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Highlights  of  GAO-15-466,  a  report  to 
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Why  GAO  Did  This  Study 

DOD’s  weapon  system  acquisition 
programs  have  a  total  estimated 
acquisition  cost  of  over  $1 .4  trillion. 
Portfolio  management  is  an  approach 
used  by  organizations  to  evaiuate, 
select,  prioritize,  and  ailocate 
resources  to  projects  that  best 
accomplish  strategic  or  organizational 
goals.  In  March  2007,  GAO 
recommended  that  DOD  implement  a 
department-wide  portfolio  management 
approach  for  weapon  system 
investments. 

Senate  Report  113-44  accompanying 
the  Nationai  Defense  Authorization  Act 
for  Fiscal  Year  2014  included  a 
provision  that  GAO  review  DOD 
processes  for  identifying  duplicative 
and  inefficient  acquisitions.  This  report 
assesses  the  extent  to  which  (1 )  DOD 
uses  portfolio  management  to  optimize 
weapon  system  investments;  (2)  DOD 
conducts  integrated  portfolio  reviews; 
and  (3)  the  military  services  conduct 
portfolio  reviews.  GAO  compared  DOD 
and  military  service  policies  and 
portfolio  reviews  with  best  practices 
and  standards  for  portfolio 
management. 

What  GAO  Recommends 

GAO  recommends  that  DOD  update  its 
portfolio  management  policy,  designate 
a  senior  official  responsible  for  its 
implementation;  conduct  annual 
portfolio  reviews  that  integrate  key 
information  from  the  requirements, 
acquisition,  and  budget  processes;  and 
invest  in  anaiytical  tools  to  support  its 
portfolio  management  efforts.  DOD 
partialiy  concurred  with  the 
recommendations.  However,  as 
discussed  in  the  report,  DOD’s  planned 
actions  will  not  fully  address  the  issues 
GAO  identified. 

View  GAO-15-466.  For  more  information, 
contact  Michael  J.  Sullivan,  (202)  512-4841  or 
sullivanm@gao.gov. 
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What  GAO  Found 

The  Department  of  Defense  (DOD)  is  not  effectiveiy  using  portfoiio  management 
to  optimize  its  weapon  system  investments,  as  evidenced  by  affordabiiity 
chaiienges  in  areas  such  as  shipbuiiding  and  potentiai  dupiication  among  some 
of  its  programs.  Best  practices  recommend  assessing  investments  coiiectiveiy 
from  an  enterprise-wide  perspective  and  integrating  requirements,  acquisition, 
and  budget  information,  but  severai  factors  inhibit  DOD’s  abiiity  to  do  so. 

•  Fragmented  governance:  DOD  has  numerous  processes,  organizations, 
and  decision  makers  to  oversee  weapon  system  investments  that  operate  in 
stove-pipes,  not  as  an  integrated  whoie.  The  requirements  and  acquisition 
processes  aiso  focus  on  individuai  programs  rather  than  assessing 
investments  coiiectiveiy,  as  best  practices  recommend. 

•  Lack  of  sustained  leadership  and  policy:  DOD  stopped  impiementing  its 
portfoiio  management  efforts  and  poiicy,  in  part  due  to  changes  in  ieadership. 
DOD’s  poiicy  is  aiso  dated,  does  not  fuiiy  reflect  best  practices,  and  does  not 
identify  an  office  with  sufficient  authority  to  impiement  it. 

•  Perceived  lack  of  decision-making  authority:  Enterprise-ievei  invoivement 
is  key  to  optimizing  investments  across  DOD  because  the  miiitary  services 
prioritize  needs  and  optimize  investments  within  their  services  rather  than 
across  the  miiitary.  Titie  10,  which  gives  the  services  responsibiiity  over 
equipping  the  force,  does  not  preciude  enterprise-ievei  influence  over  service 
investment  decisions,  but  some  DOD  officiais  said  it  iimits  their  influence. 

DOD’s  enterprise-ievei  requirements,  acquisition,  and  budgeting  communities, 
meaning  those  at  the  at  the  Office  of  the  Secretary  of  Defense,  Joint  Chiefs  of 
Staff,  and  Joint  Staff  ievei,  are  not  consistentiy  conducting  portfoiio  reviews  or 
coiiaborating  to  integrate  key  information.  As  a  resuit,  DOD  may  be  missing 
opportunities  to  better  ieverage  its  resources  and  identify  investment  priorities 
that  best  reflect  DOD-wide  needs.  Best  practices  and  portfoiio  management 
standards  state  that  organizations  shouid  conduct  reguiar  reviews  to  adjust  to 
strategic  changes,  among  other  reasons.  The  Joint  Staff,  which  is  responsibie  for 
vaiidating  warfighting  needs,  has  taken  the  most  concrete  actions  to  conduct 
portfoiio  reviews,  but  even  these  efforts  did  not  integrate  key  requirements, 
acquisition,  and  budget  information.  Requirements  and  acquisition  officiais  said 
they  iacked  the  resources,  readiiy  accessibie  data,  and  anaiyticai  toois  to 
effectiveiy  conduct  reviews.  For  exampie,  the  Joint  Staff  iacks  a  database  that 
puiis  together  current  information  to  heip  it  manage  its  portfoiios  and  has  to  reiy 
on  repeated  data  caiis,  which  are  inefficient  and  time  consuming. 

The  miiitary  services  have  conducted  reviews  more  consistentiy  than  the 
enterprise  ievei  and  their  experiences  at  the  service  ievei  may  offer  iessons  for 
DOD.  They  have  used  the  reviews  to  reduce  redundancies,  pian  for  budget 
uncertainty,  and  reaiign  resources.  Nevertheiess,  their  reviews  iack  some  of  the 
key  information  needed  to  provide  an  integrated  assessment  of  needs, 
investments,  and  resources  and  are  iimited  to  the  services’  own  programs.  A 
more  integrated  approach  to  portfoiio  reviews  at  both  the  enterprise  and  miiitary- 
service  ieveis  wouid  better  position  DOD  to  conduct  sound  investment  pianning. 
_ United  States  Government  Accountability  Office 
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U.S.  GOVERNMENT  ACCOUNTABILITY  OFFICE 


August  27,  2015 
Congressional  Committees 

The  Department  of  Defense  (DOD)  has  78  major  weapon  system 
programs  under  way  with  a  total  estimated  acquisition  cost  of  over  $1 .4 
trillion.  These  include  some  of  the  most  advanced  weapons  in  the  world. 
However,  DOD  has  not  consistently  managed  its  weapon  system 
investments  as  a  portfolio  to  ensure  they  are  strategy-driven,  affordable, 
and  balance  near-  and  long-term  needs.  Rather,  DOD  and  the  military 
services  plan  to  acquire  more  weapons  than  they  can  afford  given 
anticipated  levels  of  funding.  Furthermore,  when  multiple  services  have 
weapon  system  needs  in  common,  they  sometimes  develop  separate, 
rather  than  common  solutions,  that  result  in  inefficient,  and  in  some 
cases,  potentially  duplicative  investments. 

We  and  others  have  found  that  leading  commercial  companies  use 
portfolio  management — a  disciplined  process  that  helps  optimize 
investments  by  ensuring  organizations  have  the  right  mix  of  new  products 
that  meet  customer  needs  within  available  resources.  Portfolio 
management  focuses  on  products  collectively  at  an  enterprise  level  and 
involves  evaluating,  selecting,  prioritizing,  and  allocating  limited  resources 
to  projects  that  best  accomplish  strategic  or  organizational  goals.  It  is  also 
a  vehicle  to  make  a  wide  variety  of  decisions,  including  capability  and 
funding  trade-offs,  to  achieve  the  optimal  capability  mix  for  a  given 
investment.  In  2007,  we  recommended  that  DOD  implement  a 
department-wide  portfolio  management  approach  to  making  weapon 
system  investments.''  DOD  concurred  with  this  recommendation. 

Senate  Report  113-44  accompanying  the  National  Defense  Authorization 
Act  for  Fiscal  Year  2014  included  a  provision  that  GAO  review  DOD’s 
processes  and  procedures  for  identifying  duplicative  and  inefficient  major 
development  and  procurement  programs.^  We  examined  ways  to  reduce 


^GAO,  Best  Practices:  An  Integrated  Portfolio  Management  Approach  to  Weapon  System 
Investments  Could  Improve  DOD’s  Acquisition  Outcomes,  GAO-07-388  (Washington, 
D.C.:  Mar.  30,  2007). 

^S.  Rep.  No.  113-44,  at  143  (2013). 
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potential  duplication  and  increase  the  efficiency  and  effectiveness  of 
weapon  system  investments  through  portfolio  management.  This  report 
assesses  the  extent  to  which  (1)  DOD  uses  portfolio  management  to 
optimize  its  weapon  system  investments,  (2)  DOD  conducts  integrated 
portfolio  reviews  at  the  enterprise  level,  and  (3)  the  military  services 
conduct  integrated  portfolio  reviews  and  what,  if  any,  lessons  their 
experiences  offer  DOD. 

To  assess  the  extent  to  which  DOD  uses  portfolio  management  to 
optimize  its  weapon  system  investments,  we  compared  DOD’s  weapon 
system  investment  policies  and  decision-making  processes,  which 
include  the  requirements,  acquisition,  and  budget  processes — known 
respectively  as  the  Joint  Capabilities  Integration  and  Development 
System  (JCIDS),  Defense  Acquisition  System,  and  Planning, 
Programming,  Budgeting,  and  Execution  (PPBE)  processes — to  portfolio 
management  best  practices.  We  identified  these  best  practices  in  prior 
GAO  reports  and  conducted  a  literature  review  to  ensure  they  were  still 
current.^  To  determine  the  extent  to  which  DOD  conducts  integrated 
portfolio  reviews  at  the  enterprise  level — meaning  at  the  Office  of  the 
Secretary  of  Defense,  Joint  Chiefs  of  Staff,  and  Joint  Staff  level — we 
analyzed  enterprise-level  portfolio  reviews  and  other  portfolio-level 
analyses.  We  compared  portfolio  review  content  with  key  portfolio 
elements  that  we  identified  based  on  the  Project  Management  Institute’s 
The  Standard  for  Portfolio  Management — Third  Edition  and  best 
practices,  which  we  adapted  to  the  DOD  environment."^  We  also 
interviewed  Joint  Staff;  Office  of  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology  and  Logistics  (AT&L);  Cost  Assessment  and 
Program  Evaluation  (CAPE);  and  Office  of  the  Under  Secretary  of 
Defense  (Comptroller)  officials.  To  assess  the  extent  to  which  the  military 
services  conduct  integrated  portfolio  reviews  and  identify  potential 
lessons  for  DOD,  we  reviewed  Army,  Air  Force,  and  Navy  requirements, 
acquisition,  and  budget  policies  and  interviewed  officials  from  all  of  those 
communities.  We  also  compared  Army,  Air  Force,  and  Navy  portfolio 
reviews  with  key  portfolio  review  elements. 


^GAO-07-388. 

"^Project  Management  Institute,  Inc.  The  Standard  for  Portfolio  Management — Third 
Edition  (Newtown  Square,  Pa.:  2013). 
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We  conducted  this  performance  audit  from  November  201 3  to  August 
2015  in  accordance  with  generaiiy  accepted  government  auditing 
standards.  Those  standards  require  that  we  pian  and  perform  the  audit  to 
obtain  sufficient,  appropriate  evidence  to  provide  a  reasonabie  basis  for 
our  findings  and  conciusions  based  on  our  audit  objectives.  We  beiieve 
that  the  evidence  obtained  provides  a  reasonabie  basis  for  our  findings 
and  conciusions  based  on  our  audit  objectives. 


Background 


Weapon  system  investment  decision-making  in  DOD  is  highiy  compiex.  it 
invoives  numerous  entities,  ieveis,  and  poiicies  at  the  miiitary  service  and 
enterprise  ievei.  The  four  miiitary  services — the  Army,  Air  Force,  Navy, 
and  Marine  Corps — by  iaw  have  authority  to  organize,  train,  and  equip 
their  services.^  They  make  initiai  decisions  regarding  what  to  buy,  how  to 
buy  it,  and  how  much  it  wiii  cost.  Those  decisions  are  made  by  their 
requirements,  acquisition,  and  budget  communities  and  go  through  muiti- 
iayered  review  processes  within  the  services.  Each  service  has  its  own 
requirements,  acquisition,  and  budget  poiicies,  which  are  based  in  part  on 
enterprise-ievei  poiicies.® 

For  major  investments,  the  miiitary  services’  decisions  regarding  what  to 
buy,  how  to  buy  it,  and  how  much  to  buy  may  be  reviewed  and  approved 
or  disapproved  at  the  enterprise  ievei.  At  the  enterprise  ievei,  JCiDS 
assesses,  vaiidates,  and  prioritizes  warfighter  needs;  the  Defense 
Acquisition  System  oversees  product  deveiopment  and  procurement;^ 
and  PPBE  aiiocates  resources  taking  into  account  capabiiity  needs,  risk, 
and  affordabiiity.  Each  of  those  processes  is  governed  by  its  own  set  of 
poiicies  (See  appendix  ii  for  a  iist  of  reievant  poiicies). 


®10  U.S.C.  §§  3013,  5013,  8013,  5042. 

^Appendix  II  includes  a  list  of  key  enterprise-  and  service-level  policies  governing  the 
requirements,  acquisition,  and  budget  processes  for  weapon  system  investments. 

^The  level  of  oversight  for  acquisition  programs  varies  based  on  the  value  and  type  of 
acquisition.  Major  defense  acquisition  programs — those  estimated  to  require  an  eventual 
total  expenditure  for  research,  development,  test  and  evaluation  of  more  than  $480  million 
or,  for  procurement  of  more  than  $2.79  billion  (in  fiscal  year  2014  dollars)  for  all 
increments,  or  that  are  designated  as  such  by  the  milestone  decision  authority — can  be 
subject  to  enterprise-level  (Acquisition  Category  ID)  or  service-level  (Acquisition  Category 
1C)  oversight.  Less  expensive  programs,  known  as  Acquisition  Category  II  and  III 
programs,  are  generally  overseen  by  the  military  services.  Department  of  Defense 
Instruction  5000.02,  Operation  of  the  Defense  Acquisition  System  End.  1,  Table  1  (Jan.  7, 
2015).  (Hereinafter  referred  to  as  DODI  5000.02  (Jan.  7,  2015)). 
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Weapon  System 
Investment  Challenges 


DOD  faces  several  weapon  system  investment  planning  challenges.  First, 
DOD  plans  more  weapons  acquisition  programs  than  it  can  afford.  In  the 
Congressional  Budget  Office’s  (CBO)  most  recent  analysis  of  DOD’s  5- 
year  Future  Years  Defense  Program  (FYDP),  CBO  projected  acquisition 
costs  for  weapons  and  other  major  equipment  to  increase  by  21  percent 
between  fiscal  years  2015-2019,  and  to  peak  in  the  early  2020s  at  about 
43  percent  higher  than  in  2015.®  CBO  explained  that  the  steep  increase  in 
acquisition  costs  it  projects  beyond  the  FYDP  timeframe  shows  that  the 
classic  bow  wave  has  returned — meaning  that  DOD  is  pushing  out 
funding  needs  assuming  it  will  receive  higher  acquisition  funding  levels  in 
the  future  to  complete  its  programs.®  Similarly,  CBO  estimated  that  the 
total  costs  of  carrying  out  the  Navy’s  fiscal  year  2015  shipbuilding  plan 
would  require  one-third  more  funding  on  an  annual  basis  than  the  Navy 
has  received  in  recent  decades.''®  CBO  reported  that,  as  a  result,  the 
Navy  will  likely  have  to  purchase  substantially  fewer  ships  than  under  its 
current  plans  or  make  other  trade-offs.  In  April  2015,  we  also  raised 
affordability  issues  with  the  Joint  Strike  Fighter — DOD’s  largest  weapon 
acquisition  program.  We  estimated  the  program  will  require  an  average  of 
$12.4  billion  per  year,  which  represents  around  one-quarter  of  DOD’s 
annual  funding  for  major  weapons  programs,  over  the  next  5  years.  We 
reported  that  it  is  unlikely  the  program  will  be  able  to  sustain  such  a  high 
level  of  annual  funding  and  if  required  funding  levels  are  not  reached,  the 
program’s  procurement  plan  may  not  be  affordable.'’'' 

In  addition,  the  military  services  have  a  history  of  tailoring  their  planned 
investments  to  their  own  needs  rather  than  trying  to  find  common  ground, 
which  has  led  to  inefficiencies  and,  in  some  cases,  duplication.  More 
specifically,  the  military  services  sometimes  generate  unique  system 
requirements  that  justify  multiple  acquisition  programs  to  meet  similar 


®Congressional  Budget  Office,  Long-Term  Implications  of  the  2015  Future  Years  Defense 
Program  (Washington,  D.C.:  Nov.  2014).  CBO  assessed  the  costs  of  DOD’s  plans  using 
DOD  data  in  addition  to  CBO’s  projected  cost  growth  for  those  plans  based  on  historical 
data. 

^According  to  CBO,  bow  waves  beyond  the  FYDP  period  had  been  a  common  feature  of 
DOD’s  plans  for  many  years,  especially  during  periods  of  flat  or  declining  budgets. 

1 D 

Congressional  Budget  Office,  An  Analysis  of  the  Navy’s  Fiscal  Year  201 5  Shipbuilding 
Plan  (Washington,  D.C.:  Dec.  2014).  The  Navy’s  shipbuilding  plan  covers  fiscal  years 
2015  to  2044. 

^^GAO-F-35  Joint  Strike  Fighter:  Assessment  Needed  to  Address  Affordability  Challenges, 
GAO-15-364  (Washington,  D.C.:  Apr.  14,  2015). 
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needs.  Our  work  has  concluded  that  redundancy  or  overlapping 
capabilities  in  some  areas  may  be  desirable  or  necessary,  but  acquisition 
efforts  to  develop  similar  capabilities  can  also  result  in  the  same  capability 
gap  being  filled  twice  or  more,  lead  to  inefficient  use  of  resources,  and 
contribute  to  other  warfighting  needs  going  unfilled.  In  2012,  we  found 
potential  duplication  among  electronic  warfare  systems,  some  of  which 
DOD  has  canceled.''^  In  2013,  we  found  additional  areas  of  overlap  in  the 
roles  that  electronic  warfare  systems  were  intended  to  perform  and  that 
new  areas  of  overlap  and  potential  duplication  could  emerge  as 
investment  plans  evolved.''^  More  recently,  we  concluded  that  when 
developing  a  new  ground  radar  system,  the  Air  Force  focused  on  what 
made  its  requirements  unique  from  a  similar  Marine  Corps’  system, 
instead  of  looking  for  ways  to  leverage  the  Marine  Corps’  program.  We 
found  these  programs  to  be  potentially  duplicative. 


Portfolio  Managomont  Leading  commercial  companies  use  a  portfolio  management  approach  to 

optimize  their  investments.  They  begin  with  an  enterprise-level 
identification  and  definition  of  market  opportunities — in  other  words, 
potential  customer  wants  or  needs — and  then  prioritize  those 
opportunities  while  taking  into  account  resource  constraints.  Once  these 
companies  have  prioritized  opportunities,  they  draft  initial  business  cases 
for  alternative  product  ideas  that  could  be  developed  to  exploit  each  of 
the  highest  priority  opportunities.  Each  alternative  product  proposal  enters 
a  gated  review  process.  At  each  review,  product  proposals  are  assessed 
against  others  in  the  portfolio  weighing  corporate  resources,  established 
criteria,  competing  products,  and  the  goals  and  objectives  of  the 
organization  as  a  whole.  As  alternatives  pass  through  each  review,  the 
number  is  expected  to  decrease,  until  only  those  alternatives  with  the 
greatest  potential  to  succeed  make  it  into  the  product  portfolio.  In  effect, 
new  programs  are  born  from  portfolio  analysis  and  management.  Over 
time,  as  potential  new  products  are  identified  or  if  products  under 


^^GAO,  Airborne  Electronic  Attack:  Achieving  Mission  Objectives  Depends  on  Overcoming 
Acquisition  Chaiienges,  GAO-12-175  (Washington,  D.C.:  Mar.  29,  2012). 

^^GAO,  Next  Generation  Jammer:  DOD  Should  Continue  to  Assess  Potential  Duplication 
and  Overlap  As  Program  Moves  Forward,  GAO-13-642  (Washington,  D.C.:  Aug.  20, 

2013). 

^■’^GAO,  Ground  Radar  and  Guided  Munitions:  Increased  Oversight  and  Cooperation  Can 
Help  Avoid  Duplication  among  the  Services’  Programs,  GAO-15-1 03  (Washington,  D.C.: 
Dec.  19,  2014). 
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development  experience  problems,  companies  review  and  rebalance  their 
portfolio  of  proposed  and  existing  product  investments  based  on  those 
products  that  add  the  most  value  and  are  the  best  strategic  fit. 

More  generally,  rather  than  defaulting  to  optimizing  individual  programs, 
portfolio  management  focuses  on  selecting  the  optimum  mix  of  programs 
and  modifying  that  mix  as  needed  overtime.  Take  a  hypothetical  example 
in  which  DOD  starts  with  10  programs  and  $50  billion  to  invest.  Without 
portfolio  management,  program  managers  will  seek  to  get  the  most  that 
they  can  out  of  each  of  the  10  programs,  without  assessing  their 
aggregate  contributions  to  defense.  Using  portfolio  management,  DOD 
executives  would  look  at  different  combinations  of  and  approaches  to  the 
10  programs  to  see  what,  collectively,  would  provide  the  best  capabilities 
for  $50  billion.  This  would  enable  executives  to  decide,  for  example, 
whether  it  is  better  to  concentrate  more  investment  in  7  programs  rather 
than  fund  all  10  as  best  as  possible.  In  another  example,  if  a  program 
began  to  have  cost  or  performance  problems,  portfolio  management 
would  consider  whether  the  other  programs  in  the  portfolio  could  address 
the  requirements  of  the  problematic  program  rather  than  just  putting  more 
money  into  it. 

Portfolio  management  literature  has  identified  characteristics  and  effects 
of  investment  planning  processes  that  are  not  based  on  portfolio 
management.  These  characteristics  and  effects,  which  are  shown  in 
figure  1 ,  are  consistent  with  some  of  our  observations  on  DOD’s  weapon 
system  investment  challenges. 
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Figure  1:  What  Happens  When  Organizations  Do  Not  Use  Portfoiio  Management? 


Source:  GAO  adaptation  of  Cooper  R.G.,  Edget  S.J.,  Kleinschmidt  E.G.  Portfolio  Management  for  New  Products,  2nd  ed. 
(Cambridge,  MA:  Perseus  Publishing,  2001)  4-6.  |  GAO-15-466 


Portfolio  Managomont  in  2007,  we  identified  several  best  practices  for  portfolio  management, 

Best  Practices  which  form  the  basis  for  our  assessments  in  this  report.  Organizations 

that  follow  these  practices 

•  assess  product  investments  collectively  from  an  enterprise  level, 
rather  than  as  independent  and  unrelated  initiatives; 

•  continually  make  go/no-go  decisions  through  a  gated  review  process 
to  rebalance  portfolios  based  on  investments  that  add  the  most  value; 

•  use  an  integrated  approach  to  prioritize  needs  and  allocate  resources 
in  accordance  with  strategic  goals; 
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•  rank  and  select  investments  using  a  disciplined  process  to  assess  the 
costs,  benefits,  and  risks  of  alternative  products; 

•  empower  leadership  to  make  investment  decisions  and  hold 
leadership  accountable  for  investment  outcomes;  and 

•  provide  sustained  leadership  for  portfolio  management. 


Portfolio  Roviows  Portfolio  management  best  practices  and  the  Project  Management 

Institute’s  portfolio  management  standards  also  state  that  organizations 
should  conduct  regular  reviews  to  adjust  to  strategic  changes  or  changes 
in  the  mix  of  products  within  a  portfolio,  among  other  reasons.''^  These 
reviews  are  tools  that  organizations  use  to  plan  and  manage  their 
investments.  From  a  DOD  perspective,  portfolio  reviews  can  help 
increase  return  on  taxpayers’  investments  in  weapon  systems  in  a 
number  of  ways,  such  as:  helping  to  ensure  investments  align  with 
national  security  and  military  strategies;  prioritizing  the  most  important 
investments;  selecting  the  optimum  mix  of  investments;  identifying  and 
eliminating  unwarranted  duplication;  monitoring  programs’  health  to 
determine  whether  changes  to  the  portfolio  are  warranted;  and 
determining  whether  investments  are  affordable.  Although  there  is  not 
one  model  for  portfolio  reviews  that  fits  all  situations,  we  identified  key 
portfolio  elements,  which — if  included  in  a  review — would  integrate 
relevant  information  from  the  requirements,  acquisition,  and  budget 
communities  to  achieve  some  of  the  benefits  described  above  (see  table 
1). 


^^Project  Management  Institute,  Inc.  The  Standard  for  Portfolio  Management — Third 
Edition. 
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Table  1:  Key  Elements  to  Include  in  Portfolio  Reviews  and  How  They  May  Help  the  Portfolio  Manager 

Cross-cutting  elements 

Description  and  benefit  to  portfolio  manager 

Definition  of  the  portfolio 

Characterizes  the  portfolio  and  types  of  capabilities  it  includes. 

Goals 

Identifies  goals  for  the  portfolio  based  on  organizational  goals  and  strategy. 

Portfolio  components 

Provides  understanding  of  existing  and  planned  portfolio  components,  (e.g.,  weapon 
systems  or  weapon  system  programs)  and  their  key  attributes,  such  as  capabilities,  cost, 
time,  and  strategic  goals  they  support. 

Threats 

Identifies  key  current  and  anticipated  future  threats  to  inform  strategic  alignment  analysis. 

Strategic  alignment  analysis 

Assesses  how  portfolio  components  are  aligned  with  organizational  strategy.  As  strategies 
change,  components  may  need  to  change. 

Current  program  priorities 

Ranks  portfolio  components  based  on  pre-established  criteria.  Informs  decisions 
regarding  the  optimal  investment  mix. 

Potential  overlap  and  duplication 

Informs  decisions  regarding  whether  the  overlap  is  warranted  or  if  changes  or 
cancellations  would  ensure  better  use  of  resources. 

Recommendations  (portfolio  component 
trades,  cancellations,  truncations) 

Identifies  potential  solutions  to  problems  that  are  identified  and  investment  options  based 
on  the  team’s  analyses. 

Requirements  elements 

Capability  gaps 

Compares  current  with  desired  future  capabilities  based  in  part  on  strategic  alignment 
analysis.  Informs  investment  decisions. 

Prioritized  capability  gaps 

Ranks  capability  gaps  and  potential  solutions  using  a  disciplined  process.  Informs 
investment  decisions. 

Inventory  (capacity)  analysis 

Assesses  capacity  needs  based  on  current  and  planned  future  inventories. 

Assessment  of  whether  requirements  are 
outdated 

Identifies  divestment  opportunities  and  open  requirements  that  no  longer  need  to  be 
fulfilled  based  on  strategic  alignment  analysis. 

Risk  analysis 

Brings  attention  to  a  variety  of  risks,  such  as  relative  risk  of  various  threat  scenarios  and  of 
not  filling  or  only  partially  filling  certain  gaps. 

Acquisition  elements 

Program  health:  components’  cost, 
schedule,  and  performance  indicators 

Includes  information  on  cost,  schedule,  and  performance  of  ongoing  programs  compared 
to  their  original  business  cases.  Identifies  risks  to  dependent  programs.  Informs  decisions 
about  cancellations,  alternatives,  and  other  investment  options. 

Timeline  of  systems 

Shows  when  portfolio  components  will  be  operational  or  will  be  divested.  Can  bring 
attention  to  potential  capability  or  capacity  gaps  and  risks  for  dependent  programs. 

Dependencies  internal  or  external  to  the 
portfolio 

Identifies,  tracks,  and  manages  dependencies,  including  portfolio  component  reliance  on 
other  technology  or  product  development  programs. 

Science  and  technology  needs 

Identifies  and  tracks  science  and  technology  development  options  of  potential  benefit  to 
one  or  more  programs  in  the  portfolio. 

Budget  elements 

Total  and  relative  cost  for  key  portfolio 
components  and/or  groups  of 
components 

Assesses  relative  costs  of  portfolio  components  to  help  inform  decisions  regarding 
component  and  portfolio  affordability. 

Funding  scenarios 

Identifies  potential  funding  scenarios  and  which  portfolio  modifications  may  be  needed 
and  when. 

Source:  GAO  analysis  of  portfolio  management  best  practices,  the  Project  Management  Institute’s  The  Standard  for  Portfolio  Management-Third  Edition,  and  DOD  policies.  |  GAO-15-466 
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DOD  Is  Not  Using  an 
Integrated  Portfolio 
Management 
Approach  to  Optimize 
Its  Weapon  System 
Investments  at  the 
Enterprise  Level 


DOD  is  not  using  an  integrated  portfolio  management  approach  to 
optimize  its  weapon  system  investments  at  the  enterprise  level,  as  called 
for  in  best  practices.  DOD  lacks  the  governance  structure,  sustained 
leadership,  and  policy  to  do  so.  In  particular,  DOD’s  governance  structure 
includes  decision-making  processes  and  responsibilities  that  are  divided 
among  its  stove-piped  requirements,  acquisition,  and  budget  communities 
and  focus  largely  on  optimizing  individual  investments.  This  fragmentation 
does  not  allow  for  an  integrated  portfolio  management  approach  to 
making  investment  decisions  across  the  department.  In  addition,  the 
military  services  operate  in  stove-pipes  of  their  own  and  tend  to  optimize 
investments  for  their  service,  rather  than  for  the  military  as  a  whole.  A 
perceived  lack  of  enterprise-level  influence  over  the  services  hinders 
DOD’s  ability  to  address  this  behavior  and  optimize  investments  across 
the  department. 


DOD’s  Stove-Piped 
Governance  Structure  Is 
an  Impediment  to  Portfolio 
Management 


DOD’s  stove-piped  governance  structure  is  an  impediment  to  using  an 
integrated  portfolio  management  approach  to  optimize  weapon  system 
investments.  DOD  has  numerous  enterprise-level  processes, 
organizations,  and  decision  makers  to  oversee  its  weapon  system 
investments,  which  generally  operate  as  stove-pipes,  not  as  an  integrated 
whole.  In  addition,  the  acquisition  process  and  to  some  extent,  the 
requirements  process,  are  geared  toward  making  decisions  about 
individual  programs  rather  than  assessing  investments  collectively  from 
an  enterprise  level,  as  best  practices  suggest.  DOD  has  taken  some 
steps  to  integrate  its  requirements,  acquisition,  and  budget  processes. 
However,  these  steps  primarily  include  sending  representatives  to 
participate  in  the  other  communities’  decision-making  forums.  As  a  result, 
most  investment  decisions  get  made  on  a  piecemeal  basis  within  the 
requirements,  acquisition,  and  budget  processes.  DOD  officials  stated 
that  decision-making  forums,  such  as  the  Deputy  Secretary  of  Defense’s 
Management  Action  Group,  serve  an  integrating  function  and  make 
departmental  management  decisions  on  a  more  cross-cutting  basis,  but 
these  forums  primarily  address  issues  on  an  ad  hoc  basis.  Table  2 
describes  the  key  enterprise-level  governance  structures  that  are 
responsible  for  DOD  weapon  system  investment  decision-making. 
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Table  2:  Key  Enterprise-Level  Governance  Structures  for  Weapon  System  Investment  Decision-Making 

Weapon  system  investment 
decision-making  process 

Joint  Capabilities  Integration 
and  Development  System 

Defense  Acquisition  System 

Planning,  Programming, 
Budgeting,  and  Execution 
System 

Primary  role 

Identify,  assess,  validate,  and 
prioritize  capability  needs 

Manage  product  development 
and  procurement 

Allocate  resources,  taking  into 
account  capability  needs,  risk, 
and  affordability 

Key  decision  maker/s  and 
entities 

Chairman  of  the  Joint  Chiefs  of 
Staff;  Joint  Requirements 
Oversight  Council;  Joint 
Capabilities  Board;  Joint  Staff 

Under  Secretary  of  Defense  for 
Acquisition,  Technology  and 
Logistics 

Secretary  of  Defense;  Deputy 
Secretary  of  Defense;  Under 
Secretary  of  Defense 
(Comptroller)  and  Chief 

Financial  Officer;  Under 

Secretary  of  Defense  for  Policy; 
Director,  Cost  Assessment  and 
Program  Evaluation 

Key  responsibilities 

Identify,  assess,  approve,  and 
assign  priority  to  joint 
warfighting  requirements; 
ensure  resources  are  consistent 
with  a  requirement’s  priority; 
assess  program  alternatives 

Authorize  acquisitions  to 
proceed  based  on  cost, 
schedule,  performance,  and 
product  knowledge  attained 

Develop  overarching  program 
and  fiscal  guidance;  review 
component  budget  submissions; 
conduct  program  execution 
reviews 

Frequency 

Continuous,  needs-driven 
process 

Event-driven 

Calendar-driven 

Portfolio  construct 

Joint  Capability  Areas 

Varies  by  purpose 

Major  Force  Programs 

Source:  GAO  assessment  of  DOD  policies.  See  appendix  II  for  list  of  applicabie  policies.  |  GAO-15-466 

Note:  This  table  focuses  on  enterprise-level  governance.  There  are  numerous  decision  makers,  such 

as  service  acquisition  executives,  who  shape  and  manage  weapon  system  investments,  including 
some  major  defense  acquisition  programs,  in  the  military  services. 


Other  aspects  of  DOD’s  governance  structure  for  making  weapon  system 
investment  decisions  are  aiso  fragmented  and  inconsistent  with  an 
integrated  portfoiio  management  approach. 

•  Decision-making  responsibiiities:  DOD’s  enterprise-ievei  decision¬ 
making  responsibiiities  are  fragmented,  which  makes  it  difficuit  to 
identify  who  is  responsibie  for  investment  decisions  and  to  hoid  them 
accountabie.  According  to  portfoiio  management  best  practices, 
ieadership  shouid  be  cieariy  defined  and  heid  accountabie  for 
outcomes,  in  DOD,  weapon  system  investment  decisions  are 
dispersed  among  muitiple  decision  makers.  This  fragmentation  couid 
iead  to  conflicting  decisions  or  decisions  that  aiign  with  the  goais  of 
one  community,  but  not  those  of  another.  Severai  DOD  officiais  said 
weapon  system  investment  decision-making  is  dispersed  by  design 
and  that  naturai  tension  between  and  different  perspectives  among 
decision  makers  are  beneficiai  to  the  process.  Uitimateiy,  though. 
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these  diffuse  decision-making  responsibilities  make  it  difficult  to 
determine  who  is  empowered  to  make  enterprise-level  weapon 
system  investment  decisions  and  who  can  be  considered  to  be 
portfolio  managers.  In  our  enterprise-  and  service-level  interviews, 
some  officials  stated  that  no  one  at  the  enterprise  level  is  empowered 
to  make  investment  decisions,  and  others  responded  that  only  a 
strong  Secretary  of  Defense  or  Deputy  Secretary  Defense  can  take 
actions  required  for  effective  portfolio  management,  such  as  making 
trades  across  services  or  canceling  programs. 

•  Investment  planning  assumptions:  DOD  is  not  using  an  integrated, 
common  set  of  planning  assumptions,  such  as  warfighting  scenarios, 
to  inform  investment  decisions  across  the  department.  Best  practices 
state  that  organizations  should  rank  and  select  investments  using  a 
disciplined  process  to  assess  the  costs,  benefits,  and  risks  of 
alternative  products.  Former  senior  and  current  DOD  officials 
explained  that  it  is  difficult  to  analyze  investments  across  the  services’ 
portfolios  without  a  common  analytical  baseline  because 
organizations  use  scenarios  that  favor  their  specific  interests.  In  a 
November  2014  memo,  the  Deputy  Secretary  of  Defense  outlined 
steps  to  reinvigorate  DOD’s  support  for  strategic  analysis  efforts  and 
move  toward  a  common  starting  point  for  departmental  analysis. 
These  steps  included  Deputy  Secretary  and  Vice  Chairman  of  the 
Joint  Chiefs  of  Staff  level  review  of  scenario  selection,  key 
assumptions,  and  concepts  of  operations,  among  others  areas. 

•  Portfolio  constructs:  To  the  extent  that  the  requirements, 
acquisition,  and  budget  communities  group  investments  by  portfolio, 
they  do  so  using  different  portfolio  constructs.  According  to  portfolio 
management  best  practices,  organizations  should  use  an  integrated 
approach  to  prioritize  needs  and  allocate  resources  in  accordance 
with  strategic  goals.  Using  different  portfolio  constructs  is  a  barrier  to 
taking  an  integrated  approach.  At  the  enterprise  level,  the 
requirements  community  uses  nine  joint  capability  areas  for 
examining  warfighting  needs,  acquisition  portfolios  vary  by  military 
service,  and  budget  data  is  organized  into  eleven  major  force 
programs  for  the  FYDP.  (Table  3  lists  these  portfolios.) 
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Table  3:  Examples  of  Enterprise-Level  Portfolio  Constructs 

Portfolio  constructs 

Portfolios 

Requirements  community: 

•  Force  Support 

• 

Net-Centric 

Joint  Capability  Areas 

•  Battlespace  Awareness 

• 

Force  Protection 

•  Force  Application 

• 

Building  Partnerships 

•  Logistics 

• 

Corporate  Management  and 

•  Command  and  Control 

Support 

Acquisition  community: 

Air  Force 

Affordability  Portfolios 

•  Space  Superiority 

• 

Cyberspace  Superiority 

•  Rapid  Global  Mobility 

• 

Command  and  Control 

•  Personnel  Recovery 

• 

Air  Superiority 

•  Nuclear  Deterrence  Operations 

• 

Agile  Combat  Support 

•  Global  Precision  Attack 

• 

Building  Partnerships 

•  Global  Integrated  Intelligence  Surveillance  and 

• 

Special  Operations 

Reconnaissance 

• 

Classified  Programs 

Army 

•  Aviation 

• 

Chemical  Demilitarization 

•  Mission  Command  (Network) 

• 

Soldier 

•  Maneuver  Combat  Vehicles 

• 

Fires 

•  Air  and  Missile  Defense 

• 

Science  and  Technology 

•  Transportation 

• 

Other 

Navy 

•  Expeditionary  (Land) 

• 

Missiles 

•  Ships 

• 

Naval  Air 

Budget  community: 

•  Strategic  Forces 

• 

Central  Supply  and  Maintenance 

Major  Force  Programs 

•  General  Purpose  Funds 

• 

Training,  Medical,  and  Other 

•  Command,  Control,  Communications, 

General  Personnel  Activities 

Intelligence,  and  Space 

• 

Administration  and  Associated 

•  Mobility  Forces 

Activities 

•  Guard  and  Reserve  Forces 

• 

Support  of  Other  Nations 

•  Research  and  Development 

• 

Special  Operations  Forces 

Source:  GAO  analysis  of  DOD  policies  and  documents.  |  GAO-15-466 

DOD  attempted  to  standardize  portfolios  in  the  2006  to  2008  time  period. 
However,  a  former  senior  official  who  was  involved  in  that  effort  said  the 
mapping  was  “impossible”  and  that  there  was  organizational  resistance 
because  the  portfolios  did  not  align  with  many  decision  makers’  areas  of 
responsibility.  Many  of  the  enterprise-  and  service-level  officials  we 
interviewed  said  using  a  wide  variety  of  constructs  is  necessary  and 
sometimes  beneficial  given  the  different  roles  and  perspectives  of  the 
organizations  involved.  However,  when  they  want  to  analyze  their 
portfolios  from  another  perspective — for  example,  examining  funding 
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associated  with  joint  capability  areas — they  have  to  go  through  extensive 
mapping  exercises,  as  notionally  illustrated  in  figure  2. 


Figure  2:  Notional  Mapping  of  Portfolio  Constructs 

Requirements  community  portfolios 


Acquisition 

community 

portfolios 


L 


) 

I 


Each  portfolio 
maps  to  multiple 
other  portfolios 


Budget 

community 

portfolios 


Source:  GAO  analysis  of  DOD  policies  and  documents.  |  GAO-15-466 


DOD  Lacks  Sustained 
Leadership  to  Implement 
Portfolio  Management 


DOD  lacks  sustained  leadership  for  portfolio  management,  and  as  a 
result,  it  has  been  difficult  for  DOD  to  maintain  momentum  for  portfolio 
management-related  initiatives.  Our  prior  work  found  that  effective 
portfolio  management  requires  a  governance  structure  with  sustained 
leadership.  However,  changing  leadership  priorities  pose  challenges  to 
portfolio  management  in  DOD.  In  the  mid-to-late  2000s,  DOD  made  a 
concerted  effort  to  implement  capability  portfolio  management.  DOD 
initiated  portfolio  management  pilots  in  select  capability  areas  to  improve 
strategic  decisions  on  resource  allocation  across  programs  in  2006  and 
issued  a  directive  on  capability  portfolio  management  in  2008.''®  Soon 
afterwards,  leadership  priorities  shifted  away  from  portfolio  management. 
DOD  eliminated  the  pilots,  did  not  document  the  results  of  the  pilots,  and 
stopped  implementing  the  directive.  This  was  attributed  to  a  variety  of 


^®Department  of  Defense  Directive  7045.20,  Capability  Portfolio  Management  (Sept.  25, 
2008).  (Hereinafter  cited  as  DODD  7045.20  (Sept.  25,  2008)). 
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factors,  including  the  lack  of  a  senior  level  champion  for  the  effort  as 
political  leadership  changed,  which  can  happen  every  few  years  in  DOD. 
Officials  from  AT&L  and  the  Joint  Staff  said  that  DOD  no  longer  has  a 
champion  for  portfolio  management. 


DOD  Lacks  a  Portfolio 
Management  Policy  That 
Reflects  Best  Practices 


DOD  does  not  have  a  policy  to  guide  portfolio  management  across  the 
department  that  fully  reflects  key  best  practices.  The  policy  is  also  not 
current  and  DOD  is  not  implementing  it,  but  it  has  not  been  rescinded. 
DOD  Directive  7045.20,  Capability  Portfolio  Management,  states  that 
DOD  shall  use  capability  portfolio  management  to  advise  the  Deputy 
Secretary  of  Defense  and  the  heads  of  the  DOD  components  on  how  to 
optimize  investments  across  DOD  and  minimize  risk  in  meeting  capability 
needs.  Portfolios  are  to  be  managed  by  civilian  and  military  capability 
portfolio  managers  who  are  to  develop  capability  portfolio  strategic  plans, 
provide  cross-component  perspectives,  recommend  capability  priorities  to 
senior  decision  makers,  and  take  other  actions,  such  as  developing 
portfolio  governance  forums.  In  addition,  the  policy  attempts  to 
standardize  DOD’s  portfolio  constructs  by  aligning  portfolios  with  joint 
capability  areas. 


However,  under  the  policy,  portfolio  managers  do  not  have  sufficient 
authority  to  effectively  influence  weapon  system  investment  decisions. 
They  do  not  have  any  independent  decision-making  authority,  nor  are 
they  to  “infringe  on  any  existing  statutory  or  regulatory  authorities”  such 
as  the  military  services’  authority  under  title  10  to  organize,  train,  and 
equip  their  forces.  Instead,  their  influence  is  to  be  through  the  Deputy 
Secretary’s  Management  Action  Group,  which  is  to  use  information  from 
the  portfolio  managers  to  inform  the  Deputy  Secretary  of  Defense.  More 
generally,  the  policy  is  intended  to  operate  within  DOD’s  existing 
fragmented  organizational  and  process  framework — which  includes 
multiple  barriers  to  integrated  portfolio  management.  Table  4  includes  our 
assessment  of  how  well  DOD’s  current  portfolio  management  policy 
addresses  key  best  practices. 


^^Underthe  policy,  each  portfolio  is  to  have  two  capability  portfolio  managers — one 
military  and  one  civilian.  For  example,  military  capability  portfolio  managers  include  the 
U.S.  Strategic  Command,  Joint  Requirements  Oversight  Council,  and  Director,  Joint  Staff 
Force  Structure  Resources  and  Assessment  Directorate.  Civilian  capability  portfolio 
managers  include  the  Under  Secretaries  of  Defense  for  Intelligence;  Policy;  and 
Acquisition,  Technology,  and  Logistics. 
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Table  4:  GAO  Comparison  of  Department  of  Defense  Capability  Portfolio  Management  Policy  with  Best  Practices 


Best  practice 

Extent  met 

Comments 

Assess  product  investments  collectively 
from  an  enterprise  level,  rather  than  as 
independent  and  unrelated  initiatives 

• 

States  that  portfolio  management  will  be  used  to  optimize  investments 
across  DOD.  Requires  portfolio  managers  to  evaluate  capability  demand 
against  resource  constraints,  identify  and  assess  risks,  and  recommend 
trade-offs  within  the  context  of  their  portfolios. 

Continually  make  go/no-go  decisions 
through  a  gated  review  process  to 
rebalance  portfolios  based  on 
investments  that  add  the  most  value 

O 

Does  not  call  for  a  regular  portfolio-based  gated  review  process,  but 
portfolio  managers  are  to  inform  some  existing  decision-making 
processes. 

Use  an  integrated  approach  to  prioritize 
needs  and  allocate  resources  in 
accordance  with  strategic  goals 

O 

Existing  fragmented  processes  remain  generally  unchanged  although 
some  new,  more  integrated,  processes  are  to  be  added.  Portfolio 
managers  are  to  help  ensure  portfolios  align  with  strategic  goals. 

Rank  and  select  investments  using  a 
disciplined  process  to  assess  the  costs, 
benefits,  and  risks  of  alternative  products 

o 

Portfolio  managers  are  to  make  recommendations  about  investment 
priorities,  but  no  ranking  or  formal  processes  are  required. 

Empower  leadership  to  make  investment 
decisions  and  hold  leadership 
accountable  for  investment  outcomes 

o 

Portfolio  managers  are  not  given  decision-making  authority.  Policy  is  to  be 
applied  within  existing  organizational  structure,  with  fragmented  decision¬ 
making  authorities  and  does  not  include  provisions  to  hold  leadership 
accountable  for  investment  outcomes. 

Provide  sustained  leadership  for  portfolio 
management 

o 

Although  the  policy  attempts  to  inject  a  portfolio  perspective  into  existing 
decision-making  processes,  leadership  responsibilities  in  the  directive  are 
diffuse  and  there  is  no  one  office  responsible  for  ensuring  its 
implementation. 

Source:  GAO  analysis  of  portfolio  management  best  practices  and  DODD  7045.2,  Capability  Portfolio  Management.  |  GAO-15-466 


Legend:  •  completely  addressed;  ^  partially  addressed;  O  not  addressed. 


Officials  we  met  with  throughout  DOD  explained  that  they  are  not 
implementing  DOD’s  capability  portfolio  management  policy.  A  few 
officials  stated  that  DOD  is  implementing  aspects  of  portfolio 
management,  but  in  a  less  integrated  fashion.  Others  appeared  to  be 
unaware  of  the  policy.  Several  enterprise-level  officials  who  were  familiar 
with  the  policy  stated  that  DOD  is  not  implementing  the  directive  because 
elements,  such  as  using  a  single  set  of  portfolios,  are  impractical  and 
leadership  priorities  changed.  In  the  policy,  DOD  also  did  not  identify  an 
office  to  be  responsible  for  its  overall  implementation.  Federal  internal 
control  standards  require  that  organizations  establish  objectives  and 
clearly  define  key  areas  of  authority  and  responsibility.''®  The  Office  of  the 
Under  Secretary  of  Defense  for  Policy  (OUSD  (Policy))  issued  DOD’s 


^®GAO,  Standards  for  Internal  Control  In  the  Federal  Government,  GAO/AIM D-00-21 .3. 1 
(Washington,  D.C.:  November  1999). 
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capability  portfolio  management  directive,  however,  that  office  does  not 
have  direct  line  management  authority  over  the  organizations  responsible 
for  implementing  it.  DOD  officials  explained  that  the  only  officials  with 
authority  over  those  organizations  are  the  Deputy  Secretary  of  Defense 
and  Secretary  of  Defense.  An  official  from  OUSD  (Policy)  stated  that  they 
are  aware  that  the  policy  is  not  being  followed.  OUSD  (Policy)  is 
reviewing  the  policy  and  may  rescind  or  revise  it  in  the  future. 


Perceived  Lack  of 
Enterprise-Level  Decision- 
Making  Authority  over  the 
Services  Is  an  Impediment 
to  Portfolio  Management 


Enterprise-level  involvement  is  important  to  optimize  investments 
collectively  across  the  military  because  the  military  services  tend  to  focus 
on  optimizing  their  own  investments.  For  DOD  to  effectively  manage 
investments  at  the  enterprise  level,  it  would  need  to  exercise  its  authority 
and  influence  service-level  investment  decisions  and  make  cross-service 
trades.  Three  senior  enterprise-level  officials  explained  that  the  military 
services  focus  on  maintaining  their  portion  of  the  investment  budget  and 
tend  to  optimize  investments  for  their  own  services,  rather  than  for  the 
military  as  a  whole. 


A  perceived  lack  of  authority  over  the  military  services  is  an  impediment 
to  managing  investments  collectively  at  the  enterprise  level.  According  to 
best  practices,  portfolio  managers  should  be  empowered  to  make 
investment  decisions.  Under  title  10  the  military  services  have  the 
responsibility  for  equipping  their  forces — in  other  words,  buying  weapons. 
Enterprise-level  officials  we  interviewed  stated  that  they  think  this 
provides  primary  control  over  setting  weapon  system  investment  priorities 
to  the  services.  While  title  10  does  not  preclude  enterprise-level  influence 
over  investment  decisions,  this  influence  currently  comes  in  the  form  of 
oversight  reviews  after  the  services  have  developed  investment  plans, 
weapon  system  requirements,  and  acquisition  plans.  Joint  Staff  officials 
said  they  have  portfolio  management  responsibilities,  such  as  prioritizing 
requirements,  but  their  influence  is  limited  because  the  services  control 
the  money.  AT&L  officials  stated  that  they  have  an  oversight  role,  but 
cannot  change  the  military  services’  acquisition  priorities.  Similarly,  DOD 
Comptroller  officials  explained  that  enterprise-level  involvement  to 
optimize  investments  across  the  department  is  limited  by  title  10. 

Experiences  with  ground  radar  systems  show  some  of  the  challenges  the 
enterprise  level  has  experienced  when  trying  to  optimize  investments 
across  services.  In  2008,  the  Undersecretary  of  Defense  for  AT&L 
established  a  Radar  Joint  Analysis  Team  to  develop  a  DOD-wide 
roadmap  for  radar  technology,  development,  and  procurement  and 
consider  overlap  and  commonality  among  programs.  In  two  cases,  the 
team  encouraged  the  military  services  to  collaborate  on  developing  a 
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single  system  to  address  their  needs.  However,  the  services  had  different 
budget  priorities  and  acquisition  timelines  and  therefore  decided  to 
pursue  separate  acquisition  programs.  More  recently,  some  Office  of  the 
Secretary  of  Defense  officials  noted  there  were  similarities  between  the 
Air  Force’s  Three-Dimensional  Expeditionary  Long-Range  Radar  and  the 
Marine  Corps’  Ground/Air  Task  Oriented  Radar  requirements,  and 
questioned  the  need  for  two  separate  programs.  An  AT&L  official  said 
that  the  discussions  that  resulted  from  the  questions  were  beneficial,  but 
the  Air  Force  and  the  Marine  Corps  are  going  forward  with  separate 
programs,  in  part,  because  they  own  their  own  resources. 


DOD  Does  Not 
Conduct  Integrated 
Enterprise-Level 
Portfolio  Reviews 


DOD  does  not  conduct  enterprise-level  portfolio  reviews  that  integrate  the 
requirements,  acquisition,  and  budget  communities.  Instead,  the  Joint 
Staff,  AT&L,  and  CAPE  each  have  their  own  limited  portfolio  review 
efforts,  some  of  which  are  still  being  developed.  Conducting  portfolio 
reviews  on  a  regular  basis  can  help  optimize  portfolios.  For  example, 
reviews  can  serve  as  a  tool  to  identify  potentially  duplicative  capabilities 
or  systems  and  evaluate  trade-offs  to  address  those  inefficiencies.  DOD 
officials  said  that  limited  resources  and  a  lack  of  analytical  tools  have 
hampered  their  ability  to  conduct  integrated  portfolio  reviews  and 
portfolio-based  analyses. 


DOD  does  not  conduct  integrated  enterprise-level  portfolio  reviews. 
Instead,  the  Joint  Staff,  AT&L,  and  CAPE  have  a  fragmented  approach 
that  involves  separate,  but  limited,  portfolio  reviews  and  analyses  using 
different  portfolio  constructs.  None  of  these  reviews  and  analyses  have 
been  consistently  conducted.  Portfolio  management  best  practices  state 
that  organizations  should  conduct  regular  reviews  to  adjust  to  factors, 
such  as  strategic  changes.  Consistently  conducting  portfolio  reviews 
provides  opportunities  to  identify  and  address  inefficiencies  within 
portfolios,  and  to  adjust  and  realign  priorities  to  meet  current  needs. 

In  addition  to  not  being  conducted  in  an  integrated  manner,  the  reviews 
we  assessed  did  not  consistently  incorporate  key  portfolio  review 
elements,  which  would  result  in  a  more  integrated  portfolio  review.  Best 
practices  state  that  taking  an  integrated  approach  to  conducting  reviews 
helps  avoid  pursuing  more  investments  than  are  affordable,  balance  near- 
and  long-term  needs,  and  maximize  return  on  investment.  In  DOD’s  case, 
an  integrated  review  for  weapon  system  investments  would  involve  the 
requirements,  acquisition,  and  budget  communities  or  information  from 
those  communities  (see  figure  3). 


DOD  Does  Not  Conduct 
Integrated  Portfolio 
Reviews 
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Figure  3:  Integrated  Reviews  Would  Include  Information  from  the  Requirements,  Acquisition,  and  Budget  Communities 


Requirements  elements 

•  Capability  gaps 

•  Prioritized  capability  gaps 

•  Inventory  analysis 

•  Assessment  of  whether 
requirements  are  outdated 

•  Risk  analysis 


Acquisition  elements 

•  Program  health  indicators 

•  Timeline  of  systems 

•  Dependencies 

•  Science  and  technology 
needs 


Cross-cutting  elements 

•  Definition  of  the  portfolio 

•  Goals 

•  Portfolio  components 

•  Threats 

•  Strategic  alignment  analysis 

•  Current  program  priorities 

•  Potential  overlap  and 
duplication 

•  Recommendations 

Budget  elements 

•  Total  and  relative  cost  for 
key  portfolio  components 
and/or  groups  of 
components 

•  Funding  scenarios 


Source;  GAO  analysis  of  GAO-07-388,  PMI  Standards  on  Portfolio  Management,  and  DOD  policies.  |  GAO-1 5-466 


Joint  Staff  Reviews  to 
Date  Have  Varied  in 
Content  and  Quality 


We  assessed  five  out  of  the  six  Joint  Staff  portfolio  reviews  conducted  in 
2014  and  found  that  these  reviews  varied  widely  in  content  and  quality.  A 
Joint  Staff  official  who  coordinated  the  reviews  said  they  varied  because  it 
was  the  Joint  Staff’s  first  year  conducting  them.''®  The  Joint  Staff  initiated 
the  portfolio  reviews  to  inform  its  position  on  DOD’s  fiscal  year  2016 
budget  submission.  The  goal  of  these  reviews  was  to  identify 
redundancies,  possible  trade-offs,  investment  opportunities,  outdated 
requirements,  and  impactful  issues  for  each  portfolio.  A  Joint  Staff  official 
who  led  one  of  the  reviews  said  that  he  found  great  value  in  conducting 
the  review,  which  allowed  his  office  to  evaluate  how  the  cost  of  a 
capability  impacts  whether  other  capability  needs  within  the  portfolio 
could  be  met.  In  comparing  the  reviews  against  the  stated  goals,  all  five 
included  impactful  issues,  such  as  cost  drivers  within  a  particular  portfolio. 
We  found  that  four  reviews  included  information  on  investment 
opportunities  and  redundancies;  two  addressed  outdated  requirements; 
and  one  identified  possible  trade-offs.  Joint  Staff  officials  said  they  plan  to 
continue  conducting  portfolio  reviews  to  inform  the  budget  and  to  develop 


1 Q 

We  did  not  review  the  Battlespace  Awareness  portfolio  review  because  of  its 
classification  level,  but  we  did  interview  the  Functional  Capabilities  Board  Chair  about  the 
process  used  to  conduct  it. 
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guidance  for  the  next  round  of  reviews  to  try  to  ensure  they  are  more 
consistent. 

The  Joint  Staff  portfolio  reviews  did  not  integrate  information  from  the 
requirements,  acquisition,  and  budget  communities.  For  example,  none  of 
the  reviews  included  information  on  program  cost,  schedule,  or 
performance,  and  only  one  included  a  timeline  of  when  systems  in  the 
portfolio  would  be  fielded.  In  addition,  none  of  the  reviews  considered 
funding  scenarios  and  only  two  reviews  included  information  on  portfolio 
affordability,  such  as  details  on  funding  for  the  portfolio  over  the  next  5 
years.  By  not  incorporating  this  information  into  the  reviews,  the  Joint 
Staff  may  be  ill-equipped  to  shape  decisions  about  balancing  capability 
needs  and  portfolio  affordability  and  offer  recommendations  on  portfolio 
modifications  to  address  potential  budget  changes. 

The  Joint  Staff  recently  revised  its  JCIDS  policy  and  guidance  to  provide 
more  emphasis  on  portfolio  management. The  revised  JCIDS  manual 
states  that  Functional  Capabilities  Board  (FCB)  chairs  need  to  monitor 
activities  that  could  affect  their  portfolios  throughout  the  year  and 
reassess  capability  requirements  portfolios  as  needed. For  example, 
they  may  assess  their  portfolios  at  the  request  of  Joint  Staff  senior 
leadership,  to  inform  annual  reviews  such  as  the  budget  review,  and  to 
reassess  the  content  or  priorities  of  the  portfolio  due  to  changes  in 
strategic  guidance.  In  addition,  the  FCBs  are  to  track  acquisition 
programs’  progress  toward  satisfying  capability  requirements.  Tracking 
progress  will  help  ensure  that  the  FCBs  can  assess  changes  to  programs 
or  timelines  for  their  effect  on  their  portfolio’s  capability  needs. 


AT&L’s  Portfolio  Review 
Initiative  Has  Resulted  in 
Few  Reviews 


In  DOD’s  first  iteration  of  the  Better  Buying  Power  initiatives,  which  seek 
to  address  affordability  challenges  by  increasing  the  efficiency  of  defense 
spending,  AT&L  directed  that  portfolio  reviews  be  conducted  to  identify 
potential  redundancies;  however  few  have  been  conducted  (see  table 


'?0 

Department  of  Defense,  Manual  for  the  Operation  of  the  Joint  Capabilities  Integration 
and  Development  System  (Feb.  12,  2015). 

^VCBs  advise  the  JROC  regarding  their  requirements  portfolios,  among  other  duties. 
FCBs  are  aligned  with  joint  capability  areas,  which  are  portfolios  of  similar  capabilities. 
Each  FCB  is  chaired  by  a  general  or  flag  officer,  or  government  civilian  equivalent. 
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5).^^  AT&L’s  2010  guidance  called  for  portfolio  reviews  at  the  enterprise 
level  and  for  the  military  services  to  conduct  similar  reviews  for  lower 
dollar  acquisition  programs,  known  as  acquisition  category  (ACAT)  II  and 
III  programs.  (Appendix  III  assesses  additional  portfolio-based  analyses 
called  for  under  Better  Buying  Power.)  The  Better  Buying  Power  initiatives 
have  evolved  over  the  past  few  years.  The  most  recent  version  of  the 
guidance  still  contains  the  goal  of  eliminating  redundancy  within 
warfighter  portfolios,  but  it  is  no  longer  an  area  of  emphasis. 


Table  5:  Better  Buying  Power  Activities  Related  to  Portfolio  Reviews  and  Their  Implementation  Status 

Better  Buying  Power  activity 

Description 

Implementation  status 

Enterprise-level  portfolio  reviews  (1.0) 

The  Under  Secretary  of  Defense  for 
Acquisition,  Technology  and  Logistics  (AT&L) 
will  conduct  portfolio  reviews,  based  on  the 
Army’s  model,  at  the  joint  and  department¬ 
wide  level. 

Partially  implemented.  AT&L  has  only 
requested  reviews  of  the  Ground  Moving 
Target  Indicator  and  Integrated  Air  and 
Missile  Defense  portfolios. 

Component-level  portfolio  reviews  for 
Acquisition  Category  (ACAT)  II  and  III 
systems  (1.0) 

Conduct  portfolio  reviews  for  ACAT  II  and 

III  programs  to  identify  and  eliminate 
redundancy. 

Not  implemented.  Military  services 
determined  that  existing  ad  hoc  working 
groups  are  sufficient. 

Source:  GAO  assessment  of  Better  Buying  Power  guidance  and  implementation  status.  |  GAO-15-466 

In  2013,  AT&L  began  conducting  portfolio  reviews,  called  mission  area 
portfolio  assessments,  but  they  have  had  a  limited  focus  and  do  not 
include  some  key  information  suggested  by  portfolio  management 
standards  and  best  practices.  These  reviews  focused  largely  on 
identifying  science  and  technology  investments  needed  to  keep  up  with 
future  threats.  An  AT&L  official  said  that  the  first  set  of  reviews  included 
some  requirements,  acquisition,  and  budget  information  such  as 
identification  of  capability  gaps,  science  and  technology  needs,  and 


^^Office  of  the  Under  Secretary  of  Defense,  Acquisition,  Technology,  and  Logistics 
Memorandum:  “Better  Buying  Power:  Guidance  for  Obtaining  Greater  Efficiency  and 
Productivity  in  Defense  Spending”  (Sept.  14,  2010);  Office  of  the  Under  Secretary  of 
Defense,  Acquisition,  Technology,  and  Logistics  Memorandum:  “Implementation  Directive 
for  Better  Buying  Power  -  Obtaining  Greater  Efficiency  and  Productivity  in  Defense 
Spending”  (Nov.  3,  2010). 

^^ACAT  II  programs  are  programs  that  do  not  meet  the  criteria  for  ACAT  I  or  I A  programs, 
but  have  an  estimated  total  cost  of  more  than  $185  million  for  research,  development,  test, 
and  evaluation  or  more  than  $835  million  for  procurement  in  fiscal  year  2014  constant 
dollars.  ACAT  III  programs  are  all  other  programs  that  do  not  meet  the  criteria  for  ACAT  II 
or  above.  DOD  can  designate  programs  to  higher  ACAT  levels  based  on  special  interest. 
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affordability  analyses.  However,  the  reviews  did  not  include  other  key 
requirements  and  budget  information,  such  as  prioritized  lists  of  capability 
gaps,  outdated  requirements,  and  funding  scenarios  that  would  allow 
them  to  look  at  the  portfolio  in  an  integrated  manner  as  suggested  by 
portfolio  management  standards  and  best  practices.  An  AT&L  official 
responsible  for  the  reviews  said  that  AT&L  is  in  the  beginning  stages  of 
developing  these  reviews  and  it  is  still  determining  how  to  break  down 
larger  portfolios  to  review,  how  they  will  complement  other  portfolio 
reviews,  and  what  roles  other  Office  of  the  Secretary  of  Defense 
stakeholders  and  the  military  services  might  play  in  the  review  process. 
AT&L  initially  planned  to  conduct  these  reviews  on  an  annual  basis,  but  it 
has  not  been  able  to  do  so  due  to  time  and  resource  constraints  and  a 
decision  has  yet  to  be  made  on  whether  the  reviews  will  continue. 


CAPE  Does  Not  Conduct  At  the  direction  of  Deputy  Secretary  of  Defense,  CAPE  coordinates 

Regular  Portfolio  Reviews  strategic  portfolio  reviews  for  select  portfolios  or  issue  areas  to  inform 

budget  decisions;  however,  these  reviews  have  evolved  over  time  and  the 
topics  covered  vary  from  year-to-year.  The  original  purpose  of  the 
strategic  portfolio  reviews  was  to:  (1)  look  across  service  weapon  system 
portfolios  to  identify  redundancies  and  capability  gaps,  and  (2)  provide 
DOD  components  with  earlier  decisions  on  capability  needs,  including 
which  component  would  include  funding  in  their  budget  submission  for  the 
highest  priority  capabilities.  CAPE  has  generally  not  used  the  reviews  in 
the  way  that  they  were  originally  intended.  The  reviews  are  not 
necessarily  organized  by  portfolio  unlike  the  Joint  Staff  and  AT&L 
reviews — they  may  sometimes  address  just  one  system.  In  addition,  they 
address  more  specific,  targeted  questions  that  vary  from  year-to-year 
depending  on  what  issues  the  Deputy  Secretary  of  Defense  identifies  as 
important.  CAPE  officials  also  said  that  the  reviews  are  not  designed  to 
specifically  address  duplication  of  weapon  system  capabilities.  However, 
if  CAPE  notices  potential  duplication  in  the  course  of  its  reviews,  it  may 
propose  changes  to  programs  during  the  budget  review  process. 
According  to  the  Assistant  Secretary  of  Defense  for  Acquisition,  the  topics 
for  last  year’s  reviews  included  space,  strategic  offense,  protection  forces, 
innovation,  infrastructure,  and  readiness.  A  CAPE  official  stated  that 
some  reviews  are  more  consistent  with  the  stated  purpose  of  strategic 
portfolio  reviews  than  others.  For  example,  a  recent  review  of  the  nuclear 
enterprise  helped  prioritize  capability  needs  and  provided  the  context  for 
evaluating  investment  plans  during  the  program  budget  review  process. 
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Lack  of  Resources  and 
Analytical  Tools  Limits 
DOD’s  Ability  to  Effectively 
Conduct  Portfolio  Reviews 


The  Joint  Staff,  AT&L,  and  CAPE  reported  that  resource  limitations  have 
affected  their  ability  to  regularly  conduct  portfolio  reviews.  Despite  these 
resource  constraints,  they  have  not  coordinated  their  portfolio  reviews  to 
integrate  and  streamline  them,  although  they  have  had  conversations 
about  their  reviews  and  the  Joint  Staff  and  AT&L  have  provided  input  to 
strategic  portfolio  reviews.  We  have  previously  found  that  one  way  to 
better  manage  fragmented  activities  is  to  improve  collaboration  and 
coordination.^'^  This  includes  engaging  in  key  practices  such  as  defining 
and  articulating  common  outcomes,  agreeing  on  roles  and 
responsibilities,  and  identifying  and  addressing  needs  by  leveraging 
resources.  Without  better  coordination  and  collaboration,  DOD  will 
struggle  both  from  an  information  and  resource  perspective  to  implement 
integrated  portfolio  reviews. 

Both  AT&L  and  the  Joint  Staff  said  that  a  lack  of  readily  accessible  data 
and  analytical  tools  also  hampered  their  ability  to  effectively  conduct 
portfolio  reviews.  AT&L  officials  said  that  more  accessible  data  and  better 
tools  would  be  helpful  for  identifying  redundancies  and  for  the  mission 
area  portfolio  assessments.  Having  readily  accessible  data  and  tools  is 
essential  for  these  analyses.  For  example,  in  the  past,  lack  of  visibility  into 
certain  programs,  such  as  special  access  programs,  urgent  operational 
needs,  and  lower  ACAT  level  programs  contributed  to  potential 
unnecessary  duplication.^®  Joint  Staff  officials  said  that  past  efforts  relied 
on  repeated  data  calls,  which  were  a  drain  on  resources  and  time 
consuming.  In  addition,  one  of  the  officials  in  charge  of  a  Joint  Staff 
portfolio  review  said  that  the  Joint  Staff  does  not  have  the  authority  to 
compel  other  DOD  components  to  provide  the  necessary  information  and 
data  to  conduct  the  reviews.  Finally,  Joint  Staff  officials  report  that  they  do 
not  have  the  analytical  tools  and  dynamic  databases  to  effectively 
conduct  portfolio  reviews,  assess  potential  redundancy,  and  prioritize 
capabilities. 


^'^GAO,  Fragmentation,  Overlap,  and  Duplication:  An  Evaluation  and  Management  Guide, 
GAO-1 5-49SP  (Washington,  D.C.:  Apr.  14,  2015). 

2®See  GAO-1 5-1 03,  GAO-13-642;  GAO-12-175;  2012  Annual  Report:  Opportunities  to 
Reduce  Duplication,  Overlap  and  Fragmentation,  Achieve  Savings,  and  Enhance 
Revenue,  GAO-12-342SP  (Washington,  D.C.:  Feb.  28,  2012);  Warfighter  Support:  DOD’s 
Urgent  Needs  Processes  Need  a  More  Comprehensive  Approach  and  Evaluation  for 
Potential  Consolidation,  GAO-11-273  (Washington,  D.C.:  Mar.  1, 2011);  and  Defense 
Acquisitions:  Opportunities  Exist  to  Achieve  Greater  Commonality  and  Efficiencies  among 
Unmanned  Aircraft  Systems,  GAO-09-520  (Washington,  D.C.:  July  30,  2009). 
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The  Joint  Staff  is  currently  developing  a  database  to  provide  it  with  a 
better  analytical  tool  to  support  portfolio  management.  The  database  is  to 
track  capability  gaps  to  enable  the  Joint  Staff  to  more  easily  identify 
potential  redundancies,  prioritize  capability  gaps  and  close  out  capability 
gaps  that  have  been  addressed  or  that  no  longer  need  to  be  addressed 
because  the  threats  have  evolved.  Joint  Staff  officials  said  that  assessing 
duplication  is  challenging  because  as  investment  plans  and  capability 
needs  evolve,  new  areas  of  overlap  and  potential  duplication  can  emerge. 
They  expect  the  database  will  help  them  identify  which  programs  are 
addressing  particular  capability  gaps,  which  gaps  remain  open,  which 
gaps  are  closed,  and  which  gaps  are  outdated.  Knowing  if  there  are 
multiple  systems  with  requirements  addressing  the  same  capability  gaps 
could  help  identify  potential  duplication. 

The  Joint  Staff  has  not  yet  developed  a  formal  plan  to  address  some  of 
the  database’s  known  limitations.  First,  the  database  has  been  difficult  to 
populate  and  is  potentially  missing  information  because  it  is  dependent  on 
another  system  that  has  limited  functionality.  Second,  the  Joint  Staff  has 
had  limited  coordination  outside  of  the  immediate  office  developing  the 
database  to  determine  its  utility  to  others,  although  the  Joint  Staff  and 
AT&L  are  funding  cross-cutting  studies  to  help  refine  the  tool.  Third,  the 
Joint  Staff  has  plans  to  improve  database  usability  and  functionality,  but 
has  not  developed  a  formal  implementation  plan  to  ensure  that  it 
continues  to  improve  and  meet  the  needs  of  its  current  and  potential 
users.  Developing  such  plans  is  a  key  project  management  practice.^® 
Without  establishing  this  planning  foundation,  the  Joint  Staff  will  not  be  in 
a  sound  position  to  effectively  monitor  and  evaluate  the  implementation  of 
its  efforts  to  provide  quality  information  for  reviews. 


^®Proiect  Management  Institute,  Inc.  The  Standard  for  Program  Management,  Third 
Edition,  2013. 
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Military  Service 
Portfolio  Reviews 
Offer  Lessons  for 
DOD,  but  Could  Be 
Improved 


Most  military  services  have  investment  planning  processes  that  review 
weapon  system  portfolios  and  their  experiences  may  offer  potential 
lessons  for  DOD.  The  Army’s  capability  portfolio  reviews  have  been  held 
up  as  a  model  by  DOD  and,  in  the  past,  were  used  by  the  Army  to  re¬ 
examine  requirements  and  realign  resources.  The  Air  Force’s  investment 
plans  take  into  account  budget  uncertainty  by  examining  multiple  funding 
scenarios.  The  Navy  conducts  assessments  that  include  portfolio-based 
analysis  to  provide  an  analytical  basis  for  evaluating  budget  proposals 
against  capability  needs. However,  these  reviews  also  lacked  some  of 
the  key  acquisition,  requirements,  and  funding  information  that  would 
better  position  the  services  to  conduct  sound  investment  planning. 


Army  Re-examined 
Requirements  in  its  Initial 
Reviews  to  Inform  Trade¬ 
offs,  but  Stopped 
Conducting  Regular 
Portfolio  Reviews 


Strengths:  The  Army  did 

•  conduct  portfolio  reviews  to  re-examine 
requirements  in  portfolio  context; 

•  assess  cost-effectiveness  of  capabilities 
and  long-term  capacity  needs;  and 

•  recommend  changes  to  weapon  system 
programs,  including  cancellations. 

Weaknesses:  The  Army  did  not 

•  consistently  conduct  portfolio  reviews; 

•  regularly  include  a  majority  of  key  portfolio 
review  elements  in  its  reviews;  and 

•  incorporate  review  guidance  into  policy. 


The  Army  has  conducted  portfolio  reviews,  which  could  offer  lessons  for 
DOD,  especially  in  the  area  of  re-examining  capability  needs  from  a 
portfolio  perspective.  The  Army’s  portfolios  are  generally  aligned  with  the 
U.S.  Army  Training  and  Doctrine  Command’s  Centers  of  Excellence,  such 
as  aviation,  although  the  portfolio  groupings  have  changed  over  time  and 
increased  in  number  from  nine  to  eleven.  The  Secretary  of  the  Army 
initiated  capability  portfolio  reviews  in  December  2009.  The  purpose  of 
the  reviews  is  to,  among  other  things,  ensure  that  funds  are  programmed, 
budgeted,  and  executed  against  valid  requirements  and  informed 
alternatives  and  to  revalidate  portfolios  based  on  strategy.  Combatant 
Commanders  needs,  and  affordability.  The  reviews  are  led  by  the 
requirements  community  and  involve  the  acquisition  and  budget 
communities.  The  process  involves  multiple  working  group  meetings  and 
then  a  series  of  briefings  involving  increasingly  higher  level  officials. 

These  culminate  in  a  session  with  Army  senior  leadership  to  identify 
portfolio  priorities  and  other  recommendations.  A  former  senior  official 
who  spearheaded  the  reviews  stated  that  the  Army  originally  planned  to 
review  each  portfolio  individually  on  an  annual  basis. 

Some  Army  capability  portfolio  reviews  resulted  in  changes  to  weapon 
system  investments,  including  program  cancellations,  as  a  result  of  re¬ 
examining  requirements  and  capabilities  from  a  portfolio  perspective.  For 


^^The  Marine  Corps  is  a  military  service  under  the  Department  of  the  Navy.  The  Navy  has 
direct  oversight  over  the  Marine  Corps’  higher-dollar  value  programs. 

90 

The  Army’s  original  capability  portfolios  were  air  and  missile  defense;  aviation;  combat 
vehicle  modernization;  engineer  mobility  and  counter  mobility;  intelligence,  surveillance, 
and  reconnaissance;  network  modernization;  precision  fires;  soldier  systems;  and  tactical 
wheeled  vehicles. 
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example,  a  2010  Precision  Fires  review,  which  examined  the  Army’s 
balance  of  high-end  precision  munitions  and  lower-end  near-precision 
munitions,  led  to  several  changes.  The  review  determined  that  a 
proposed  precision  munitions  system — the  Non-Line-of-Sight  Launch 
System — would  not  provide  cost-effective  capabilities.  The  Army 
canceled  the  program  and  relied  on  another  system  to  provide  similar 
capabilities  at  lower  cost.  Based  on  that  review,  the  Army  also  decided  to 
reduce  the  number  of  certain  munitions  rounds  due  to  cost  and  other 
factors.^®  AT&L  held  up  this  review  as  a  model  in  its  first  iteration  of  the 
Better  Buying  Power  initiatives,  which  stated  that  enterprise-level  and 
component-level  portfolio  reviews  were  to  be  conducted  with  an  eye 
toward  identifying  redundancies  and  savings. 

Although  the  Army’s  capability  portfolio  reviews  have  been  repeatedly 
cited  as  a  potential  model  for  DOD,  they  did  not  consistently  include  key 
requirements,  acquisition,  and  budget  portfolio  review  elements  that  could 
have  yielded  more  comprehensive  reviews.  For  example,  most  of  these 
reviews  did  not  prioritize  capability  gaps,  incorporate  information  on 
science  and  technology  needs,  or  assess  investment  options  for  multiple 
potential  funding  scenarios.  As  a  result,  the  reviews  did  not  position  the 
Army,  for  example,  to  proactively  identify  strategies  to  deal  with  varying 
funding  scenarios.  Results  of  our  analysis  of  how  well  the  Army’s  six 
portfolio  reviews  from  2012  incorporated  key  review  elements  are 
included  in  table  6. 


oq 

Excalibur  and  Accelerated  Precision  Mortar  Initiative. 
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Table  6:  Key  Portfolio  Review  Elements  Included  in  the  Army’s  Six  Capability  Portfolio  Reviews  from  2012 

Key  portfolio  review  elements 

Included 

Comments 

Cross-cutting  elements 

Definition  of  the  portfolio 

• 

Listed  categories  of  systems  that  fit  within  the  portfolio. 

Goals 

• 

Provided  goals  for  the  portfolio  but  did  not  explicitly  link  them  to 
elements  of  strategy. 

Portfolio  components 

• 

Listed  key  systems  or  programs. 

Threats 

O 

Listed  key  threats;  one  listed  anticipated  evolution  of  threats. 

Strategic  alignment  analysis 

o 

Did  not  map  capabilities  to  specifics  of  strategy. 

Current  program  priorities 

o 

Most  ranked  or  listed  priorities. 

Potential  overlap  and  duplication 

o 

One  review  specified  whether  programs  had  been  assessed  for 
redundancy. 

Recommendations  (portfolio  component  trades, 
cancellations,  truncations) 

• 

Types  of  recommendations  varied,  but  ranged  from 
recommendation  of  further  assessment  to  specific 
recommendations  of  trades  and  risks  associated  with  trades. 

Requirements  elements 

Capability  gaps 

• 

Most  provided  high-level  description  of  gaps. 

Prioritized  capability  gaps 

o 

One  review  ranked  priorities,  including  some  gaps. 

Inventory  (capacity)  analysis 

o 

Some  reviews  included  long-term  capacity  plans. 

Assessment  of  whether  requirements  are  outdated 

o 

Some  reviews  assessed  whether  requirements  should  be  re¬ 
evaluated  or  if  they  had  not  yet  been  validated. 

Risk  analysis 

o 

Most  reviews  did  not  show  evidence  of  comprehensive  risk 
analysis. 

Acquisition  elements 

Program  health:  components’  cost,  schedule,  and 
performance  indicators 

o 

Two  reviews  provided  program  health  indicators. 

Timeline  of  systems 

• 

Reviews  provided  timelines  of  programs,  including 
modernization,  sustainment,  and  divestment  information. 

Dependencies  internal  or  external  to  the  portfolio 

o 

No  reviews  listed  dependencies. 

Science  and  technology  needs 

o 

One  review  included  needs  and  existing  efforts  to  address 
them. 

Budget  elements 

Total  and  relative  cost  for  key  portfolio  components 
and/or  groups  of  components 

• 

Most  addressed  relative  costs  of  portfolio  components  over 
time. 

Funding  scenarios 

o 

Reviews  did  not  present  multiple  funding  scenarios. 

Source:  GAO  analysis  of  Army  documents.  |  GAO-15-466 

Legend:  O  0  to  2  reviews  addressed  them;  ^  3  to  4  reviews  addressed  them;  •  5  to  6  reviews 
addressed  them. 
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Furthermore,  the  Army’s  more  recent  portfolio  review  efforts  have  not 
been  consistent,  in  part  because  of  a  lack  of  current  guidance.  A  201 0 
memorandum  issued  by  the  Secretary  of  the  Army  provided  direction  for 
the  reviews,  although  it  has  not  been  incorporated  into  Army  guidance. 
The  memo  was  to  be  valid  for  1  year  after  issuance  and  called  for  high 
level  Army  officials  to  make  recommendations  during  that  year  regarding 
whether  to  continue  conducting  the  reviews,  incorporate  them  into  other 
Army  requirements  and  acquisition  processes,  develop  new  processes 
that  combine  requirements  validation  with  portfolio  reviews,  or  terminate 
the  reviews.  The  Standard  for  Internal  Control  in  the  Federal 
Government — Third  Edition,  notes  that  agency  policies  should  be  clearly 
documented. Despite  the  instructions  in  the  memo.  Army  officials  stated 
that  memo  has  not  been  updated,  nor  have  the  details  of  the 
memorandum  been  incorporated  into  Army  guidance,  although  the  Army 
has  continued  to  conduct  the  reviews. 

In  the  past  two  years,  the  Army  has  only  conducted  reviews  for  4  of  its  1 1 
portfolios.  Some  of  these  more  recent  reviews  were  less  comprehensive 
than  earlier  reviews.  For  example,  both  the  March  2014  Movement 
Maneuver  and  the  September  2013  Combating  Weapons  of  Mass 
Destruction  capability  portfolio  reviews  do  not  include  key  budget 
elements,  such  as  funding  scenarios  and  charts  showing  the  relative 
costs  of  portfolio  components  over  time.  Without  such  information,  review 
participants  cannot  readily  determine  whether  programs  are  affordable  in 
their  broader  portfolio  context  and  what  changes  to  the  portfolio  would  be 
warranted  given  varying  potential  funding  scenarios. 

The  Army  budget  community  has  recently  focused  efforts  on  a  different 
review,  the  Long-Range  Investment  Requirements  Analysis  (LIRA).  This 
analysis  produces  an  assessment  of  planned  spending  for  weapon 
systems  by  portfolio  over  30  years.  The  Army  conducts  the  analysis 
annually  to  inform  the  Army’s  Program  Objective  Memorandum,  the  final 
product  of  the  programming  phase  of  the  PPBE  process.  The  analysis 
includes  funding  information  spanning  key  weapon  systems’  lifecycles 
from  initial  research  and  development  to  divestiture.  According  to  Army 
budget  officials,  the  LIRA  process  enables  the  Army  to  assess  levels  of 
risks  and  analyze  if  the  investment  decisions  they  make  now  will  be 
affordable  in  the  long  term,  and  determine  the  resources  required  for 


^°GAO,  Standards  for  Internal  Control  In  the  Federal  Government,  GAO/AIM D-00-21 .3. 1 
(Washington,  D.C.:  November  1999). 
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each  portfolio.  Army  requirements  officials  said  that  the  LIRA  is  a  broader, 
longer-term  examination  of  the  Army’s  budget  than  the  portfolio  reviews. 
The  Army’s  portfolio  reviews  provide  some  information  to  inform  the 
LIRAS  although  the  two  are  conducted  separately. 


Air  Force  Portfolio 
Reviews  Address  Most 
Key  Review  Elements,  but 
Lack  Certain  Acquisition 
Information 


Strengths:  The  Air  Force  did 

•  conduct  portfolio  reviews  to  identify  the 
best  short-  and  long-term  investments; 

•  assess  needs  and  priorities  given  multiple 
risk  and  funding  scenarios; 

•  develop  a  template  to  help  ensure 
consistent  review  content;  and 

•  identify  trades  within  and  across 
portfolios. 

Weaknesses:  The  Air  Force  did  not 

•  include  certain  key  portfolio  review 
elements  in  its  reviews  and 

•  develop  guidance  specific  to  the  reviews. 


The  Air  Force  also  conducts  portfolio  reviews  that  could  offer  lessons  for 
DOD  due  to  their  comprehensiveness  and  consideration  of  multiple  risk 
and  funding  scenarios.  The  Air  Force  began  conducting  these  reviews, 
called  core  function  support  plans,  in  2010.^^  The  Air  Force  conducts  the 
reviews  for  12  Air  Force  “core  functions.”  The  purpose  of  the  reviews  is  to 
establish  the  best  investment  options  for  capabilities  in  the  near-  to  long¬ 
term.  Core  function  teams,  which  are  led  by  “core  function  lead 
integrators,”  perform  the  reviews.  The  teams  identify  and  prioritize  top- 
level  gaps  and  requirements  within  their  portfolios  and  the  plans  help 
inform  capability  development  and  the  planning  and  programming  phases 
of  the  PPBE  process.  The  teams  may  include  requirements  and  budget 
personnel,  but  membership  varies.  The  acquisition  community  is  involved 
when  it  comes  time  to  integrate  the  plans.  The  Air  Force  has  a  policy  that 
describes  its  strategic  planning  system,  but  it  is  dated  and  does  not 
provide  guidance  on  the  core  function  support  planning  process.  The  Air 
Force  is  in  the  process  of  updating  this  policy. 

One  strength  of  the  Air  Force  reviews,  which  was  different  from  the  Army 
and  Navy  reviews,  was  their  assessment  of  priorities  and  needs  given 
multiple  funding  and  risk  scenarios.  The  core  function  teams  assess 
operational  risk  using  multiple  warfighting  scenarios.  Based  on  these 
analyses,  they  identify  needs  and  priorities  given  different  funding 
scenarios,  which  can  be  a  good  technique  for  evaluating  trade-offs.  In  the 
plans  we  reviewed,  one  of  these  funding  scenarios  reflected  “zero  real 
growth,”  a  flat  budget  that  accepted  whatever  risk  level  would  result  from 
that  funding  level.  The  other  funding  level  was  relatively  unconstrained 
and  reflected  the  funding  needed  to  conduct  operations  with  moderate 
risk.  Air  Force  core  function  team  officials  also  said  they  have  used 


O'! 

Core  function  support  plans  were  previously  called  core  function  master  plans.  The  Air 
Force  conducted  other  types  of  reviews  prior  to  this  effort.  The  Air  Force  did  not  develop 
core  function  support  plans  in  2013  to  inform  its  fiscal  year  2016  budget  request  because 
of  DOD  debates  about  funding  levels  related  to  the  Budget  Control  Act  of  201 1 ,  Pub.  L. 
No.  112-25,  which,  among  other  things,  amended  the  Balanced  Budget  and  Emergency 
Deficit  Control  Act  of  1985  to  establish  limits  on  discretionary  spending  for  fiscal  years 
2012  through  2021. 
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additional  funding  levels  to  develop  core  function  support  plans,  such  as 
zero  real  growth  minus  5  or  10  percent.  Based  on  these  scenarios,  the 
team  creates  prioritized  lists  of  capabilities,  and  sometimes  potential  cuts 
or  offsets,  to  inform  budget  decisions.  The  core  function  support  planning 
process  has  led  to  decisions  to  reduce  tactical  airlift  capacity,  minimize 
facilities  modernization  and  renovation,  and  increase  munitions 
procurement. An  Air  Force  official  responsible  for  integrating  the  plans 
explained  that  trades  have  been  made  both  within  and  across  portfolios. 
In  order  to  reduce  risk  in  one  core  function  it  is  often  necessary  to  accept 
additional  risk  in  another. 

The  Air  Force’s  core  function  support  plans  could  potentially  serve  as  a 
model  for  DOD,  but  they  lack  a  few  key  elements  that  could  contribute  to 
a  more  integrated  review.  We  found  that  the  Air  Force’s  reviews  included 
a  wide  variety  of  key  portfolio  review  elements,  but  lacked  discussions  of 
potential  redundancies  and  included  limited  information  on  the  health  or 
status  of  key  acquisition  programs.  The  assessments’  content  was  also 
relatively  consistent,  in  part,  because  the  Air  Force  provides  a  template 
for  the  plans. Results  of  our  analysis  of  portfolio  reviews  from  2012  are 
included  in  table  7. 


oo 

More  specific  details  are  classified.  Core  function  support  plans  and  many  of  their  details 
are  classified  at  the  SECRET  level. 

oo 

In  one  case,  a  core  function  support  plan  we  analyzed  deviated  from  the  template.  The 
Air  Force  Space  Command  directed  the  plan  to  include  more  funding  in  its  zero  growth 
scenario  than  other  core  function  plans. 
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Table  7:  Key  Portfolio  Review  Elements  Included  in  Nine  of  the  Air  Force’s  2012  Core  Function  Support  Plans 

Key  portfolio  review  elements 

Included 

Comments 

Cross-cutting  elements 

Definition  of  the  portfolio 

• 

Provided  descriptions  of  categories  of  systems  that  fit  within  the 
portfolio. 

Goals 

• 

Provided  goals  for  the  portfolio  and  linked  them  to  elements  of 
strategy. 

Portfolio  components 

• 

Included  lists  of  applicable  programs. 

Threats 

• 

Described  key  threats  and  most  described  their  anticipated  evolution. 

Strategic  alignment  analysis 

• 

Provided  high  level  analysis  of  existing  and  needed  capabilities  in  the 
context  of  various  operational  scenarios. 

Current  program  priorities 

• 

Listed  priority  capabilities,  sometimes  for  limited  contexts. 

Potential  overlap  and  duplication 

O 

Did  not  include  any  discussion  of  overlap  or  redundancy. 

Recommendations  (portfolio  component 
trades,  cancellations,  truncations) 

• 

Presented  a  range  of  options  given  varying  funding  scenarios. 

Requirements  elements 

Capability  gaps 

• 

Most  listed  unfunded  requirements. 

Prioritized  capability  gaps 

o 

Categories  of  capabilities  ranked;  some  reviews  ranked  gaps  in  the 
context  of  priorities  if  more  funding  were  received. 

Inventory  (capacity)  analysis 

• 

Included  long-term  capacity  analyses. 

Assessment  of  whether  requirements  are 
outdated 

o 

Did  not  include  outdated  requirements. 

Risk  analysis 

• 

Included  multiple  analyses  of  operational  and  force  management  risk. 

Acquisition  elements 

Program  health:  components’  cost,  schedule, 
and  performance  indicators 

o 

Reviews  included  limited  information  related  to  portfolio  health,  such 
as  cost  and  some  performance  information. 

Timeline  of  systems 

• 

High  level  timeline  reflected  in  capacity  analyses. 

Dependencies  internal  or  external  to  the 
portfolio 

o 

Most  only  listed  dependencies  in  relation  to  other  portfolios. 

Science  and  technology  needs 

• 

Provided  detailed  information  on  science  and  technology  investment 
needs. 

Budget  elements 

Total  and  relative  cost  for  key  portfolio 
components  and/or  groups  of  components 

• 

Addressed  relative  costs  of  portfolio  components  over  time. 

Funding  scenarios 

• 

Reviews  presented  moderate  risk  force  and  zero  real  growth  funding 
scenarios. 

Source:  GAO  analysis  of  Air  Force  documents.  |  GAO-15-466 


Legend:  O  0  to  3  reviews  addressed  them;  ^  4  to  6  reviews  addressed  them;  •  7  to  9  reviews 
addressed  them. 

Note;  Of  the  Air  Force’s  12  Core  Function  Support  Plans  from  2012,  we  reviewed  the  9  that  focused 
on  weapon  system  investments. 
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Navy  Conducts  Analytical 
Assessments  to  Inform 
Navy-wide  Investment 
Plans 


Strengths:  The  Navy  did 

•  conduct  Navy-wide  assessments  to 
balance  resources  with  needs; 

•  recommend  changes  to  programs,  such 
as  terminations  and  deferrals;  and 

•  develop  guidance  for  the  assessments. 

Weaknesses:  The  Navy  did  not 

•  conduct  in-depth  reviews  for  its  individual 
portfolios  and 

•  include  several  key  portfolio  review 
elements  in  its  reviews. 


The  Navy  does  not  conduct  in-depth  reviews  for  its  individual  portfolios, 
but  it  does  conduct  annual  analytical  assessments  at  the  Navy  enterprise- 
level  to  inform  the  budget  process  that  could  offer  lessons  for  DOD. 

These  analytical  assessments,  called  for  under  the  Navy’s  Warfighting 
Capability,  Capacity,  and  Wholeness  policy,  evaluate  proposed  budgets 
in  the  context  of  capability  priorities  and  risks  and  provide 
recommendations  to  leadership  to  inform  resource-allocation  decisions. 
They  identify  near-,  mid-,  and  long-term  capability  and  capacity  gaps  and 
provide  quantitative  and  qualitative  information  to  support  trades  among 
Navy  programs.  The  Navy’s  analytical  arm  in  the  Office  of  the  Chief  of 
Naval  Operations  leads  these  assessments  and  is  to  integrate  information 
from  the  requirements  and  budget  communities  within  the  Navy  when  it 
conducts  them.  The  acquisition  community  is  generally  not  involved  in 
these  reviews.  The  process  includes  four  major  assessments  (see  figure 
4). 


Figure  4:  Navy  Warfighting  Capabiiity,  Capacity,  and  Whoieness  Process 


Capability  assessment: 


Integrates  information 
from  several  other  capability 
assessments;  the  other 
assessments  cover, 
among  other  things,  identified 
capability  gaps,  solutions 
to  those  capability  gaps, 
strategic  readiness  of  Navy 
capability  areas,  and  level 
of  funding  necessary  to  fully 
meet  a  particular  requirement. 


Front-end  assessment: 


Serves  as  the  vehicle  to 
identify  capability  gaps  and 
excesses,  areas  that  require 
additional  investment,  and 
areas  where  the  Navy  can 
take  additional  risk. 


Warfighting  support  and 
capability  plan: 


Translates  gaps  identified 
in  the  front-end  assessment 
into  feasible  and  analytically 
informed  programmatic 
options;  presents  a  fiscally 
informed  and  balanced 
overview  of  recommended 
capability  and  capacity 
changes  from  the  previous 
budget  year. 


Integrated  program 
assessment: 


Evaluates  Navy  programs 
across  all  resource  sponsors 
to  assess  risk  and  provides  a 
final  opportunity  to  realign 
programs  prior  to  finalizing 
the  budget  submission. 


Source;  GAO  analysis  of  the  Navy’s  Warfighting  Capability,  Capacity,  and  Wholeness  Assessments  policy.  |  GAO-15-466 


^'’^Department  of  the  Navy,  OPNAV  Instruction  3050.25,  Warfighting  Capabiiity,  Capacity, 
and  Whoieness  Assessments  (July  30,  2012). 
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From  a  Navy-wide  perspective,  the  Navy  has  used  its  assessments  to 
balance  its  available  resources  against  its  warfighting  needs  by  making 
trades  to  fit  its  priorities  within  its  overall  top  line.  The  assessments  we 
reviewed  included  recommendations  regarding  program  cancellations, 
truncations,  and  deferrals.  For  example,  as  a  result  of  a  more  recent 
assessment,  the  Navy  decided  to  end  production  of  the  Joint  Standoff 
Weapon.  Navy  officials  said  that  its  analytical  arm  determined  that  the 
Navy  could  truncate  production  because  it  had  produced  enough  of  the 
weapon  and  had  another  system  in  development  that  could  provide  the 
capability  in  the  future.  The  Navy’s  assessments  also  included 
information  on  capability  gaps  and  the  timeline  of  systems  to  meet  those 
gaps  by  portfolio  or  capability  area.  This  information  is  used  to  develop 
the  front-end  assessment,  which  includes  areas  where  the  Navy  should 
invest  additional  resources  to  address  priority  gaps.  According  to  Navy 
officials,  the  front-end  assessment  is  one  of  the  most  important  products 
of  the  assessment  because  it  identifies  areas  of  focus  for  the 
requirements  community  and  may  identify  areas  of  overlap  or  duplication. 
The  officials  said  that  they  can  identify  when  more  than  one  requirements 
office  has  proposed  a  solution  for  the  same  capability  gap. 

While  the  Navy’s  assessments  are  intended  to  provide  an  analytical  basis 
for  evaluating  budget  proposals  against  capability  needs;  they  do  not 
include  an  in-depth  review  of  each  portfolio  and  do  not  include  key 
information  from  the  requirements,  acquisition,  and  budget  communities. 
We  assessed  the  Navy’s  Warfighting  Capability,  Capacity,  and 
Wholeness  assessments  conducted  to  develop  its  fiscal  year  2014 
budget  request  and  found  that  the  assessments  reflected  some  key 
portfolio  review  elements  (see  table  8).  However,  the  assessments  also 
did  not  include  a  number  of  other  key  portfolio  review  elements  and,  for 
some  elements,  the  information  was  not  portfolio-based.  By  not  including 
such  information,  the  Navy  is  missing  an  opportunity  to  focus  on  portfolio- 
specific  challenges,  such  as  the  trade-offs  that  might  be  needed  to 
address  the  affordability  of  its  shipbuilding  portfolio. 
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Table  8:  Key  Portfolio  Review  Elements  Included  in  the  Navy’s  Fiscal  Year  2014  Warfighting  Capability,  Capacity  and 
Wholeness  Assessments 

Key  elements  of  a  portfolio  review 

Included 

Comments 

Cross-cutting  elements 

Definition  of  the  portfolio 

O 

Did  not  fully  define  portfolios,  but  listed  building  blocks  for  each  capability 
area. 

Goals 

O 

Did  not  list  portfolio  goals. 

Portfolio  components 

• 

Listed  components  within  each  capability  area. 

Threats 

• 

Listed  threats  and  scenarios.  Included  a  timeline  of  when  threats  begin  to 
affect  operational  risks. 

Strategic  alignment  analysis 

o 

Did  not  align  portfolio  components  to  organizational  strategy. 

Current  program  priorities 

o 

Did  not  list  current  program  priorities. 

Potential  overlap  and  duplication 

• 

Referred  to  a  system  that  could  be  canceled  because  another  system  met 
the  Navy’s  needs 

Recommendations  (portfolio  component 
trades,  cancellations,  truncations) 

• 

Recommended  Navy-wide  areas  for  additional  investment  and  potential 
offsets,  including  decreased  funding,  terminations,  and  truncations. 

Requirements  elements 

Capability  gaps 

• 

Included  some  high  priority  capability  gaps. 

Prioritized  capability  gaps 

o 

Did  not  rank  capability  gaps. 

Inventory  (capacity)  analysis 

• 

Included  current  and  future  availability  of  quantities  for  some  investments. 

Assessment  of  whether  requirements  are 
outdated 

o 

Did  not  assess  whether  requirements  are  outdated. 

Risk  analysis 

• 

Included  risk  analyses  by  capability  area. 

Acquisition  elements 

Program  health:  components’  cost, 
schedule,  and  performance  indicators 

• 

Included  some  program  health  indicators,  such  as  cost. 

Timeline  of  systems 

• 

Included  a  timeline  of  systems  by  capability  area. 

Dependencies  internal  or  external  to  the 
portfolio 

o 

Did  not  list  dependencies  internal  or  external  to  the  portfolio. 

Science  and  technology  needs 

o 

Did  not  include  science  and  technology  needs. 

Budgeting  elements 

Total  and  relative  cost  for  key  portfolio 
components  and/or  groups  of 
components 

o 

Included  one  Navy-wide  analysis  for  some,  but  not  all,  investments. 

Funding  scenarios 

o 

Did  not  include  funding  scenarios. 

Source:  GAO  analysis  of  Navy  documents.  |  GAO-15-466 

Legend:  •  included;  ^  partially  included;  and  O  not  included 

Note:  Navy  acquisition  officials  stated  that  they  address  some  of  key  portfolio  review  elements 
through  the  acquisition  process,  including:  assessment  of  whether  requirements  are  outdated, 
dependencies  internal  and  external  to  the  portfolio,  science  and  technology  needs,  and  funding 
scenarios. 
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Conclusions 


DOD  operates  as  a  joint  force  on  the  battlefield,  yet  still  does  not  use  an 
integrated  approach  to  optimizing  its  weapon  system  investments.  The 
military  services  develop  investment  plans  that  reflect  their  own  priorities. 
Enterprise-level  reviews  of  these  plans  and  acquisition  programs  are 
stove-piped  within  the  requirements,  acquisition,  and  budgeting 
communities.  Further,  oversight  processes  largely  focus  on  individual 
programs.  As  a  result,  DOD  continues  to  craft  investment  plans  that 
reflect  military  service  preferences  and  are  not  affordable  over  the  long¬ 
term  and  default  to  optimizing  for  and  addressing  problems  in  individual 
programs  instead  of  focusing  on  portfolios  of  programs  that  might  provide 
greater  military  capability  at  lower  risk  or  cost. 

Addressing  these  challenges  through  strategic  investment  planning  and 
portfolio  management  is  important  today  given  the  constrained  budget 
environment,  quickly  evolving  threats,  and  uncertainty  about  the  types  of 
conflicts  the  United  States  may  face  in  the  future.  But  DOD  currently  lacks 
the  governance  structure,  policy,  and  decision-making  authorities  to 
implement  the  more  strategic  and  integrated  approach  to  weapon  system 
investment  embodied  in  portfolio  management  best  practices.  Completely 
breaking  down  the  stove  pipes  between  the  requirements,  acquisition, 
and  budget  communities;  standardizing  portfolio  constructs;  and 
empowering  enterprise-level  portfolio  managers  to  drive  investment 
decisions  would  require  a  fundamental  restructuring  of  how  DOD  does 
business — this  is  unrealistic  in  the  short  run  and  would  be  challenging 
even  in  the  long  run.  Nevertheless,  DOD  can  lay  the  foundation  for  better 
portfolio  management  by  taking  some  incremental  steps  that  have  the 
potential  to  improve  the  efficiency  and  effectiveness  of  its  weapon  system 
investment  planning  and  management  processes. 

DOD  has  an  opportunity  to  make  meaningful  changes  to  its  portfolio 
management  efforts  if  senior  leadership  makes  a  commitment  and  invests 
the  resources  to  do  so.  Several  elements  stand  out  as  candidates  for 
immediate  action.  First,  DOD’s  current  capability  portfolio  management 
policy  is  dated,  senior  leadership  commitment  has  waned,  and,  as  a 
result,  the  policy  is  not  being  implemented.  Reforming  this  policy  to  better 
reflect  best  practices  is  essential  to  bridging  the  stovepipes  in  DOD’s 
requirements,  acquisition,  and  budget  processes  and  moving  DOD 
toward  a  more  integrated  approach  to  planning  and  managing  its  weapon 
system  investments.  Senior  leadership  with  direct  line  management 
authority  over  the  organizations  responsible  for  implementing  any  policy 
will  also  be  needed  if  the  outcomes  from  any  renewed  portfolio 
management  effort  will  be  different  from  those  in  the  past.  Next,  if  DOD  is 
to  move  toward  a  weapon  system  investment  plan  that  is  affordable. 
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strategy-driven,  and  balanced  between  near-term  and  long-term  needs, 
then  portfolio  managers  will  need  better  data  and  analytical  tools  to 
assess  threats  and  capability  needs  and  make  trade-offs  between  groups 
of  acquisition  programs  that  can  address  them.  Finally,  as  DOD  has 
recognized  in  its  Better  Buying  Power  initiatives,  the  military  services  offer 
potential  lessons  for  DOD’s  portfolio  management  efforts.  The  military 
services  have  used  portfolio  reviews  and  other  assessments  to  reduce 
redundancies,  realign  resources,  produce  more  analytically  rigorous 
budget  proposals,  and  plan  for  budget  uncertainty.  However,  even  these 
efforts  can  be  improved  by  making  sure  they  reflect  key  portfolio  review 
elements  from  across  the  requirements,  acquisition,  and  budget 
communities. 


Rpr'nmmpnHptinn«s  for  DOD’s  use  of  portfolio  management  for  its  weapon  system 

investments  and  ensure  that  its  investment  plans  are  affordable,  strategy- 
Executive  Action  driven,  balance  near-  and  long-term  needs,  and  leverage  efforts  across 

the  military  services,  we  are  making  two  recommendations  for  executive 
action. 

First,  to  provide  a  solid  foundation  for  future  portfolio  management  efforts 
at  the  enterprise-level,  we  recommend  that  the  Secretary  of  Defense 
revise  DOD  Directive  7045.2  on  Capability  Portfolio  Management  in 
accordance  with  best  practices  and  promote  the  development  of  better 
tools  to  enable  more  integrated  portfolio  reviews  and  analyses  of  weapon 
system  investments.  Key  elements  of  this  recommendation  would  include 

•  designating  the  Deputy  Secretary  of  Defense  or  some  appropriate 
delegate  responsibility  for  implementing  the  policy  and  overseeing 
portfolio  management  in  DOD; 

•  requiring  annual  enterprise-level  portfolio  reviews  that  incorporate  key 
portfolio  review  elements,  including  information  from  the 
requirements,  acquisition,  and  budget  processes; 

•  directing  the  Joint  Staff,  AT&L,  and  CAPE  to  collaborate  on  their  data 
needs  and  develop  a  formal  implementation  plan  for  meeting  those 
needs  either  by  building  on  the  database  the  Joint  Staff  is  developing 
for  its  analysis  or  investing  in  new  analytical  tools;  and 

•  incorporating  lessons  learned  from  military  service  portfolio  reviews 
and  portfolio  management  activities,  such  as  using  multiple  risk  and 
funding  scenarios  to  assess  needs  and  re-evaluate  priorities. 
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Second,  to  help  ensure  the  military  services’  portfolio  reviews  are 
conducted  regularly  and  effectively  integrate  information  from  the 
requirements,  acquisition,  and  budget  communities,  the  Secretary  of 
Defense  should  direct  the  Secretaries  of  the  Army,  Navy,  and  Air  Force  to 
update  or  develop  policies  that  require  them  to  conduct  annual  portfolio 
reviews  that  incorporate  key  portfolio  review  elements,  including 
information  from  the  requirements,  acquisition,  and  budget  processes. 


Agency  Comments 
and  Our  Evaluation 


We  provided  a  draft  of  this  report  to  DOD  for  comment.  In  its  written 
comments,  which  are  reprinted  in  full  in  appendix  IV,  DOD  partially 
concurred  with  our  two  recommendations.  However,  as  discussed  below, 
DOD’s  planned  actions  will  not  fully  address  the  issues  we  raised  in  this 
report. 

DOD  partially  concurred  with  our  first  recommendation  to  revise  DOD 
Directive  7045.20  on  Capability  Portfolio  Management  in  accordance  with 
best  practices  and  promote  the  development  of  better  tools  to  enable 
more  integrated  portfolio  reviews  and  analyses  of  weapon  system 
investments.  In  its  response,  DOD  agreed  with  the  need  to  further 
develop  portfolio  management  tools,  ensure  access  to  authoritative  data, 
and  incorporate  lessons  learned  by  others  performing  portfolio 
management.  However,  DOD  stated  that  other  aspects  of  our 
recommendation  were  redundant  to,  and  would  conflict  with  other 
processes  and  activities  in  place  to  perform  portfolio  management.  DOD’s 
response  presumes  that  its  existing  portfolio  management  processes  and 
activities  have  been  effective  and  fails  to  acknowledge  that  they  have  not 
successfully  addressed  weapon  system  investment  challenges,  such  as 
affordability  issues  in  certain  portfolios  and  potential  duplication  among 
some  programs.  We  disagree  that  existing  processes  and  activities  have 
been  effective.  Further,  we  continue  to  believe  that  our  recommendation 
to  revamp  DOD’s  portfolio  management  policy  to  align  with  best 
practices,  establish  clear  responsibility  for  its  implementation,  and 
conduct  annual  portfolio  reviews  would  provide  the  foundation  for 
improved  weapon  system  investment  planning  and  management.  DOD’s 
specific  comments  on  portfolio  management  policy,  leadership,  and 
annual  portfolio  reviews  follow. 

•  Policy:  DOD  does  not  plan  to  revise  its  capability  portfolio 
management  policy  in  accordance  with  best  practices,  as  we 
recommended,  but  rather  stated  the  policy  should  be  rescinded  and 
stakeholders  directed  to  participate  in  portfolio  management  through 
its  requirements,  acquisition,  and  budget  processes.  In  addition,  DOD 
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stated  that  recent  revisions  to  its  requirements  and  acquisition  policies 
vastly  improve  the  integration  of  these  processes.  We  continue  to 
believe  that  having  an  enterprise-level  portfolio  management  policy 
that  is  consistent  with  best  practices  is  necessary  to  provide  a  solid 
foundation  for  future  portfolio  management  efforts.  Rescinding,  rather 
than  revising,  its  enterprise-level  portfolio  management  policy  and 
relying  on  its  requirements,  acquisition,  and  budgeting  process  to  fill- 
in  the  policy  gaps  could  reinforce  the  stove-piped  governance 
structure  that  we  found  to  be  an  impediment  to  integrated  portfolio 
management. 

•  Sustained  leadership:  DOD  does  not  plan  to  designate  the  Deputy 
Secretary  of  Defense  or  an  appropriate  delegate  responsibility  for 
overseeing  portfolio  management,  as  we  recommended.  DOD 
responded  that  the  Deputy  Secretary  of  Defense  already  makes 
portfolio  management  decisions  in  the  context  of  DOD’s  budget 
process  and  that,  as  long  as  these  decisions  are  informed  by  the 
requirements  and  acquisition  communities,  they  are  sufficient  to 
adjust  portfolios  to  meet  the  military’s  needs.  We  continue  to  believe 
that  sustained  enterprise-level  leadership  for  implementing  portfolio 
management  processes  is  needed  to  support  sound  portfolio 
management  decisions  by  the  Deputy  Secretary  of  Defense  or  others. 
Our  review  found  that  key  DOD  portfolio  management-related  efforts, 
including  some  designed  to  inform  the  budget  process  and  some 
called  for  in  guidance,  have  lacked  sustained  leadership  and  been 
inconsistently  implemented.  If  the  Deputy  Secretary  of  Defense  or  an 
appropriate  delegate  assumed  leadership  for  portfolio  management,  it 
could  help  ensure  portfolio  management  policy  as  well  as  other 
portfolio  management  efforts  are  more  consistently  implemented 
throughout  the  department. 

•  Annual  portfolio  reviews:  DOD  does  not  plan  to  require  annual 
enterprise-level  portfolio  reviews  that  integrate  key  portfolio  review 
elements  from  the  requirements,  acquisition,  and  budget  processes, 
as  we  recommended.  DOD  responded  that  portfolio  reviews  would  be 
redundant  to  annual  reviews  already  conducted  through  existing 
departmental  processes.  For  example,  DOD  responded  that  the 
annual  Capability  Gap  Assessment  process  called  for  under  the 
requirements  process  serves  as  an  annual  portfolio  review  and  that 
AT&L  leads  reviews  of  long-term  science  and  technology  plans  to 
help  frame  future  capability  portfolios.  DOD  also  stated  that  it  uses 
Capability  Gap  Assessments  during  the  budget  process  to  ensure  that 
military  service  budget  submissions  are  consistent  with  the 
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warfighters’  capability  needs.  We  continue  to  believe  that  annual 
portfolio  reviews  that  incorporate  key  portfolio  review  elements  would 
provide  DOD  a  more  integrated  and  comprehensive  perspective  on  its 
weapon  system  investment  plans  than  its  current  processes  provide. 
The  processes  DOD  described  in  its  response  largely  reflect  its  stove- 
piped  governance  structures  where  the  responsibilities  for  aspects  of 
portfolio  management  are  dispersed  among  the  requirements, 
acquisition,  and  budget  communities.  Further,  our  review  found  that 
portfolio  reviews  were  inconsistently  conducted,  did  not  address  all 
portfolios,  or  only  involved  parts  of  the  organization.  As  a  result,  we 
continue  to  believe  that  updating  DOD’s  portfolio  management  policy 
to  require  annual  integrated  enterprise-level  portfolio  reviews  is  the 
best  way  to  ensure  DOD  has  the  information  it  needs  to  craft  weapon 
system  investment  plans  that  are  affordable,  balance  near-  and  long¬ 
term  needs,  and  maximize  return  on  investment. 

DOD  also  partially  concurred  with  our  second  recommendation  to  update 
or  develop  policies  that  require  the  military  services  to  conduct  their  own 
annual  portfolio  reviews  that  incorporate  key  portfolio  review  elements 
from  the  requirements,  acquisition,  and  budget  processes.  However, 

DOD  did  not  indicate  that  it  would  take  any  action  to  address  it.  Instead, 
DOD  responded  that  the  services’  budget  processes  and  Office  of  the 
Secretary  of  Defense’s  review  of  the  services’  budgets  meet  the  intent  of 
our  recommendation.  We  disagree.  Our  review  found  that  military  service 
level  portfolio  reviews,  including  Air  Force  and  Navy  reviews  closely 
linked  to  their  budget  processes,  did  not  always  contain  key  portfolio 
review  elements  or  were  not  consistently  conducted.  We  continue  to 
believe  that  updating  or  developing  policies  to  require  the  services  to 
conduct  annual  portfolio  reviews  incorporating  information  from  the 
requirements,  acquisition,  and  budget  processes  would  help  ensure  the 
services  conduct  regular,  integrated  reviews,  which  in  turn  should  yield 
benefits,  such  as  prioritizing  the  most  important  investments  and  selecting 
the  optimum  investment  mix. 
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We  are  sending  copies  of  this  report  to  the  appropriate  congressional 
committees;  the  Secretary  of  Defense;  the  Secretaries  of  the  Army,  Navy, 
and  Air  Force;  the  Commandant  of  the  Marine  Corps;  and  other 
interested  parties.  This  report  will  also  be  available  at  no  charge  on 
GAO’s  website  at  http://www.gao.gov. 

If  you  or  your  staff  have  any  questions  concerning  this  report,  please 
contact  me  at  (202)  512-4841  or  by  e-mail  at  sullivanm@gao.gov. 

Contact  points  for  our  Offices  of  Congressional  Relations  and  Public 
Affairs  may  be  found  on  the  last  page  of  this  report.  Key  contributors  to 
this  report  are  listed  in  appendix  V. 


Michael  J.  Sullivan 

Director,  Acquisition  and  Sourcing  Management 
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List  of  Committees 


The  Honorable  John  McCain 
Chairman 

The  Honorable  Jack  Reed 
Ranking  Member 
Committee  on  Armed  Services 
United  States  Senate 

The  Honorable  Thad  Cochran 
Chairman 

The  Honorable  Richard  J.  Durbin 
Ranking  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
United  States  Senate 

The  Honorable  Mac  Thornberry 
Chairman 

The  Honorable  Adam  Smith 
Ranking  Member 
Committee  on  Armed  Services 
House  of  Representatives 

The  Honorable  Rodney  Frelinghuysen 
Chairman 

The  Honorable  Pete  Visclosky 
Ranking  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
House  of  Representatives 
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Our  objectives  were  to  assess  the  extent  to  which  (1)  the  Department  of 
Defense  (DOD)  uses  portfolio  management  to  optimize  its  weapon 
system  investments,  (2)  DOD  conducts  integrated  portfolio  reviews  at  the 
enterprise  level  to  optimize  their  weapon  system  investments,  and  (3)  the 
military  services  conduct  integrated  portfolio  reviews  to  optimize  their 
weapon  system  investments  and  their  experiences  offer  potential  lessons 
for  DOD. 

To  address  our  first  objective,  we  reviewed  commercial  portfolio 
management  best  practices  from  GAO-07-388  Best  Practices:  An 
Integrated  Portfolio  Management  Approach  to  Weapon  System 
Investments  Could  Improve  DOD’s  Acquisition  Outcomes  and  reviewed 
portfolio  management-related  literature  from  peer  reviewed  journals, 
books,  and  trade  publications  to  determine  which,  if  any,  additional 
practices  should  be  added.''  Based  on  this  review,  we  determined  that  no 
changes  to  the  previously  identified  practices  were  needed.  We 
compared  these  best  practices  with  DOD’s  weapon  system  investment 
processes,  which  include  the  requirements,  acquisition,  and  budget 
processes.  We  analyzed  relevant  DOD  policies  and  statutes,  including: 
DOD’s  portfolio  management  policy,  DOD  Directive  7045.20  Capability 
Portfolio  Management;  requirements  policies,  including  Chairman  of  the 
Joint  Chiefs  of  Staff  Instruction  3170.01 1  Joint  Capabilities  Integration  and 
Development  System  and  the  Manual  for  the  Operation  of  the  Joint 
Capabilities  Integration  and  Development  System;  acquisition  policies, 
including  DOD  Directive  5000.01  The  Defense  Acquisition  System  and 
DOD  Instruction  5000.02  Operation  of  the  Defense  Acquisition  System; 
the  budget  policy,  DOD  Directive  7045.14  The  Planning,  Programming, 
Budgeting,  and  Execution  (PPBE)  Process;  and  relevant  sections  of  title 
10  of  the  United  States  Code.  We  reviewed  relevant  prior  GAO  reports 
and  DOD  memoranda,  including  the  November  2014  memorandum  from 
the  Deputy  Secretary  of  Defense  on  the  support  for  strategic  analysis 
process  and  the  July  2014  memorandum  from  the  Vice  Chairman  of  the 
Joint  Chiefs  of  Staff  on  the  support  for  strategic  analysis  process.  We  also 
compared  DOD’s  capability  portfolio  management  policy  with  portfolio 
management  best  practices  to  evaluate  the  extent  to  which  it  reflects 
those  practices.  To  describe  previous  capability  portfolio  management 
efforts  in  the  mid-to-late  2000s  and  the  extent  they  are  being 


^GAO,  Best  Practices:  An  Integrated  Portfolio  Management  Approach  to  Weapon  System 
Investments  Could  Improve  DOD’s  Acquisition  Outcomes,  GAO-07-388  (Washington, 
D.C.:  Mar.  30,  2007). 
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implemented  today,  we  reviewed  relevant  documents,  including  a 
September  2006  memorandum  from  the  Deputy  Secretary  of  Defense  on 
capability  portfolio  management  test  cases  and  military  service 
requirements,  acquisition,  and  budget  policies.  Based  on  our  assessment 
of  these  policies  and  additional  documentation,  we  also  identified 
potential  impediments  to  DOD’s  use  of  portfolio  management  to  optimize 
investments  across  the  department.  To  inform  all  assessments  for  this 
objective,  we  interviewed  officials  from  DOD  enterprise-  and  military 
service-level  requirements,  acquisition,  and  budget  communities, 
including: 

•  Requirements:  Force  Structure,  Resource  and  Assessment 
Directorate,  J-8,  and  Directorate  for  Intelligence,  J-2  for  the  enterprise 
level;  Army  Capabilities  Integration  Center  and  Headquarters, 
Department  of  the  Army  G-3/5/7  Capability  Portfolio  Review  and 
Integration;  Directorate  of  Resources;  Air  Force  Office  of  the  Director 
of  Operational  Capability  Requirements,  A5R;  the  Air  Combat 
Command;  Office  of  the  Chief  of  Naval  Operations,  N803,  Joint  & 
Urgent  Requirements;  Office  of  the  Chief  of  Naval  Operations,  N96, 
Surface  Warfare;  and  Marine  Corps,  Office  of  the  Deputy 
Commandant,  Combat  Development  and  Integration.^ 

•  Acquisition:  Office  of  the  Under  Secretary  of  Defense  for  Acquisition, 
Technology  and  Logistics  (AT&L)  for  the  enterprise  level;  Office  of  the 
Assistant  Secretary  of  the  Army  for  Acquisition,  Logistics,  and 
Technology;  Office  of  the  Assistant  Secretary  of  the  Air  Force  for 
Acquisition;  Office  of  the  Assistant  Secretary  of  the  Navy  for 
Research,  Development,  and  Acquisition;  and  Marine  Corps  Systems 
Command. 

•  Budget:  Office  of  the  Secretary  of  Defense,  Cost  Assessment  and 
Program  Evaluation  (CAPE)  and  Office  of  the  Under  Secretary  of 
Defense  Comptroller  [OUSD(C)]  for  the  enterprise  level;  Directorate  of 
Resources,  Army  G-8,  Force  Development;  Air  Force  Division  Chief, 
Planning,  Programming,  and  Requirements  Division,  Air  Combat 
Command;  the  Office  of  the  Chief  of  Naval  Operations,  N81, 
Integration  and  Force  Structure  and  the  Office  of  the  Chief  of  Naval 
Operations,  Fiscal  Management,  N82  and  Office  of  the  Assistant 


^Some  offices  represent  more  than  one  community.  For  example.  Air  Combat  Command 
represented  both  the  requirements  and  budget  communities. 
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Secretary  of  the  Navy  for  Financial  Management  and  Comptroller;  and 
Marine  Corps  Office  of  Programs  and  Resources. 

To  address  our  second  objective,  we  reviewed  relevant  guidance  and 
documents  to  identify  enterprise-level  portfolio  reviews  and  other 
portfolio-level  analyses  that  DOD  components  conduct  and  interviewed 
DOD’s  enterprise-level  requirements,  acquisition,  and  budget 
communities.  We  analyzed  standards  in  the  Project  Management 
Institute’s  The  Standard  for  Portfolio  Management — Third  Edition  and 
GAO  best  practices  and  compared  them  with  DOD  policies.  Based  on  this 
analysis,  we  developed  a  list  of  key  portfolio  elements  which,  if  included 
in  portfolio  reviews,  could  help  integrate  key  portfolio  management 
information  from  the  requirements,  acquisition,  and  budget 
communities — information  that  is  needed  for  an  integrated  approach  to 
prioritizing  needs  and  allocating  resources  in  accordance  with  strategic 
goals.  We  compared  the  Joint  Staff  Functional  Capability  Board  portfolio 
reviews’  content  with  these  key  portfolio  review  elements.^  To  assess 
whether  DOD  has  conducted  affordability  analyses  in  accordance  with 
DOD  Instruction  5000.02  and  DOD’s  Better  Buying  Power  guidance,  we 
reviewed  six  Defense  Acquisition  Board  briefings  for  programs  that  held 
an  acquisition  or  milestone  review  from  January  to  June  2014.  To  assess 
whether  DOD  has  conducted  portfolio  reviews  and  other  portfolio-based 
analyses  to  identify  and  eliminate  potential  redundancies  in  accordance 
with  DOD’s  Better  Buying  Power  guidance,  we  interviewed  Joint  Staff, 
AT&L,  and  military  service  officials,  and  reviewed  relevant  guidance.  We 
also  interviewed  Joint  Staff,  AT&L,  and  CAPE  officials  to  determine  the 
extent  to  which  the  three  offices  have  collaborated  on  these  portfolio 
reviews  and  portfolio-based  analyses,  and  to  determine  whether  they  had 
the  necessary  data,  resources,  and  analytical  tools  to  effectively  conduct 
reviews. 

To  address  our  third  objective,  we  reviewed  Army,  Air  Force,  and  Navy"^ 
requirements,  acquisition,  and  budget  policies  to  determine  the  extent  to 
which  they  required  portfolio  reviews.  We  analyzed  Army  and  Air  Force 


^We  did  not  review  the  Battlespace  Awareness  Functional  Capability  Board  portfolio 
review  because  of  its  classification  level. 

^Although  the  Navy  has  direct  oversight  over  the  Marine  Corps’  higher-dollar  value 
programs,  the  Marine  Corps  maintains  overall  responsibility  for  managing  its  weapon 
system  portfolios.  The  Marine  Corps  is  developing  portfolio  management  guidance,  which 
will  require  portfolio  reviews.  The  Marine  Corps  has  not  yet  finalized  the  guidance. 
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portfolio  reviews,  and  the  Navy’s  analytical  assessments  by  comparing 
their  contents  with  key  portfolio  review  elements.  The  military  service 
assessments  we  evaluated  include: 

•  Army:  the  six  capability  portfolio  reviews  the  Army  conducted  in  2012, 
including  Combat  Vehicles;  Precision  Fires;  Sustainment 
(Transportation);  Aviation;  Assured  Mobility  and  Protection;  and 
Mission  Command;  and  the  two  most  recent  unclassified  reviews 
provided  by  the  Army,  including  Combating  Weapons  of  Mass 
Destruction  from  2013  and  Movement  and  Maneuver  from  2014; 

•  Air  Force;  of  its  12  core  function  support  plans  from  2012  that 
informed  the  Air  Force’s  fiscal  year  2015  president’s  budget  request, 
we  reviewed  the  nine  that  had  weapon  system  centric  portfolios, 
including  Global  Precision  Attack;  Space  Superiority;  Air  Superiority; 
Global  Integrated  Intelligence,  Surveillance,  and  Reconnaissance; 
Command  and  Control;  Rapid  Global  Mobility;  Nuclear  Deterrence 
Operations;  Personnel  Recovery;  and  Special  Operations; 

•  Navy:  four  assessments  that  were  part  of  the  Navy’s  Warfighting 
Capability,  Capacity,  and  Wholeness  process  to  inform  the  Navy’s 
Program  Objective  Memorandum  for  fiscal  year  2014,  including 
Capability  Assessments;  Front-end  Assessments;  Warfighting 
Support  and  Capability  Plan;  and  Integrated  Program  Assessment. 

We  also  interviewed  officials  from  all  four  military  services’  requirements, 
acquisition,  and  budget  communities  to  determine  how  they  conduct 
these  assessments,  the  role  of  the  assessments  in  their  weapon  system 
investment  decision-making,  and  the  results  of  the  assessments.  Based 
on  our  assessment  of  these  policies,  reviews,  and  interviews  with  service 
officials,  we  identified  the  strengths  and  weaknesses  of  these  reviews  and 
the  potential  lessons  they  may  offer  DOD. 
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Enterprise-Level  Policies  Department  of  Defense,  Manual  for  the  Operation  of  the  Joint  Capabilities 

Integration  and  Development  System  (Feb.  12,  2015). 

Chairman  of  the  Joint  Chiefs  of  Staff  Instruction  3170.011,  Joint 
Capabilities  Integration  and  Development  System  (JCIDS)  (Jan.  23, 
2015). 

Chairman  of  the  Joint  Chiefs  of  Staff  Instruction  5123. 01G,  Charter  of  the 
Joint  Requirements  Oversight  Council  (JROC)  (Feb.  12,  2015). 

Department  of  Defense  Directive  5000.01 ,  The  Defense  Acquisition 
System  (May  12,  2003). 

Department  of  Defense  Instruction  5000.02,  Operation  of  the  Defense 
Acquisition  System  (Jan.  7,  2015). 

Department  of  Defense  Directive  7045.14,  The  Planning,  Programming, 
Budgeting,  and  Execution  (PPBE)  Process  (Jan.  25,  2013). 

Chairman  of  the  Joint  Chiefs  of  Staff  Instruction  8501. 01 B,  Chairman  of 
the  Joint  Chiefs  of  Staff,  Combatant  Commanders,  Chief,  National  Guard 
Bureau,  and  Joint  Staff  Participation  in  the  Planning,  Programming, 
Budgeting  and  Execution  Process  (Aug.  21 , 2012). 


Military  Service-Level 
Policies 

Apuiy  Army  Regulation  1-1,  Planning,  Programming,  Budgeting  and  Execution 

System  (Jan.  30,  1994). 

Army  Training  and  Doctrine  Command  Regulation  71-20,  Concept 
Development,  Capabilities  Determination,  and  Capabilities  Integration 
(Jun.  28,  2013). 

Army  Regulation  71-9,  Warfighting  Capabilities  Determination  (Dec.  28, 
2009). 

Army  Regulation  70-1 ,  Army  Acquisition  Policy  (Jul.  22,  201 1 ). 
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Department  of  the  Army  Pamphlet  70-3,  Army  Acquisition  Procedures 
(Apr.  1 , 2009). 

Air  Force  Air  Force  Policy  Directive  10-6,  Capabiiity  Requirements  Deveiopment 

(Nov.  6,2013). 

Air  Force  Instruction  10-601,  Operationai  Capabiiity  Requirements 
Deveiopment  {How.  6,  2013). 

Air  Force  Policy  Directive  63-1 ,  Integrated  Life  Cycle  Management  (Jul.  3, 
2012). 

Air  Force  Instruction  63-101/20-101,  Integrated  Life  Cycle  Management 
(Mar.  7,  2013). 

Air  Force  Policy  Directive  65-6,  Financial  Management:  Budget  (May  1 , 
1998). 

Air  Force  Policy  Directive  90-1 1 ,  Strategic  Planning  System  (Mar.  26, 
2009). 

[\l3vy  Secretary  of  the  Navy  Manual  M-5000.2,  Department  of  the  Navy: 

Acquisition  and  Capabilities  Guidebook  (May  9,  2012). 

Secretary  of  the  Navy  Instruction  5000. 2E,  Department  of  the  Navy 
Implementation  and  Cperation  of  the  Defense  Acquisition  System  and  the 
Joint  Capabilities  Integration  and  Development  System  (Sept.  1 , 201 1 ). 

Chief  of  Naval  Operations  Instruction  3050.25,  Warfighting  Capability, 
Capacity,  and  Wholeness  Assessments  (Jul.  30,  2012). 
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In  addition  to  its  portfolio  review  efforts,  the  Department  of  Defense 
(DOD)  has  called  for  other  portfolio-based  analyses  to  support  acquisition 
decisions  for  individual  acquisition  programs.  These  analyses  are  to 
review  the  program  in  the  context  of  its  corresponding  portfolio.  The 
Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology  and 
Logistics  (AT&L)  called  for  these  portfolio-based  analyses  to  ensure  the 
affordability  of  its  weapon  systems  programs  and  to  eliminate  programs 
that  are  potentially  duplicative  of  other  systems — two  areas  where  we 
have  identified  challenges.  However,  the  Assistant  Secretary  of  Defense 
for  Acquisition  said  that  AT&L  has  not  implemented  the  latter  of  these 
efforts  because  it  lacks  readily  accessible  data,  resources,  and  tools  to  do 
so. 

AT&L  requires  the  military  services  and  other  DOD  components  to 
conduct  portfolio-based  affordability  analyses  under  Better  Buying  Power 
and  in  its  acquisition  instruction;  however,  AT&L  has  not  reviewed  the 
quality  of  the  analyses  or  used  them  to  inform  acquisition  decisions.  We 
also  found  that  these  efforts  are  only  partially  being  implemented  (see 
table  9).  Better  Buying  Power  guidance  from  2010  and  2012  recommends 
portfolio-based  affordability  analyses  to  inform  acquisition  program 
milestone  decisions.''  Subsequently,  AT&L  required  affordability  analysis 
for  acquisition  programs  under  its  primary  acquisition  policy — DOD 
Instruction  5000.02 — as  a  tool  to  promote  responsible  and  sustainable 
investment  decisions.^  This  policy  states  that  the  purpose  of  the 
affordability  analysis  is  to  avoid  starting  or  continuing  programs  that  DOD 
cannot  produce  and  support  within  reasonable  expectations  for  future 
budgets.  The  military  services  are  to  consider  the  long-term  affordability 
of  a  program  within  the  context  of  its  portfolio  to  inform  the  program’s 
milestone  decisions.  The  portfolios  to  be  used  for  these  analyses  were 
provided  by  the  services  and  are  generally  different  than  the  portfolios 
used  for  requirements  and  budget  purposes  at  the  enterprise-wide  level. 
AT&L  officials  said  they  have  not  monitored  the  military  services’ 


^Office  of  the  Under  Secretary  of  Defense,  Acquisition,  Technology,  and  Logistics 
Memorandums:  “Implementation  Directive  for  Better  Buying  Power  -  Obtaining  Greater 
Efficiency  and  Productivity  in  Defense  Spending”  (Nov.  3,  2010);  “Better  Buying  Power 
2.0:  Continuing  the  Pursuit  for  Greater  Efficiency  and  Productivity  in  Defense  Spending” 
(Nov.  13,  2012);  and  “Implementation  Directive  for  Better  Buying  Power  2.0  -  Achieving 
Greater  Efficiency  and  Productivity  in  Defense  Spending”  (Apr.  24,  2013). 

^Department  of  Defense  Instruction  5000.02,  Operation  of  the  Defense  Acquisition 
System,  Enel.  8  (Jan.  7,  2015). 
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compliance  with  conducting  the  analyses  or  reviewed  the  quality  of  the 
analyses  because  the  responsibility  for  conducting  them  falls  to  service 
leadership. 


Table  9:  Better  Buying  Power  Activities  Related  to  Affordability  Analyses  and  Their  Implementation  Status 


Better  Buying  Power  activity 

Description 

Implementation  status 

Portfolio  investment  analyses  to  derive 
affordability  caps  Acquisition  Category 
(ACAT)  1  programs  (2.0  and  1 .0) 

DOD  components  develop  30-40  year  portfolio 
affordability  analyses  to  develop  life-cycle 
affordability  constraints.  These  are  to  inform 
acquisition  decisions.  Requirements, 
acquisition,  and  budget  offices  are  to  inform 
the  analyses. 

Partially  implemented.  Components 
provided  some  portfolio  analyses;  some 
have  covered  shorter  timeframes. 

Portfolio  affordability  analyses  for  lower 
ACAT  systems  (2.0  and  1.0) 

Same  as  above.  Milestone  decision  authority  is 
at  the  service  level. 

Partially  implemented.  Service  acquisition 
officials  stated  they  recently  began 

reviewing  these  analyses. 


Source:  GAO  assessment  of  Better  Buying  Power  guidance  and  implementation  status.  |  GAO-15-466 


In  our  review  of  defense  acquisition  board  briefings  from  January  through 
June  2014,  we  found  that  5  of  6  programs  that  held  reviews  included  an 
affordability  analysis.  However,  two  of  these  analyses  did  not  look  at 
affordability  over  the  recommended  30-  to  40-year  period,  and  for  the 
three  in  which  the  portfolio  exceeded  the  total  obligation  authority  in  any 
given  year,  there  was  no  discussion  of  possible  trades  to  make  the 
portfolio  affordable.  According  to  the  Assistant  Secretary  of  Defense  for 
Acquisition,  statute  charges  the  military  services  with  equipping  the  force, 
so  it  is  the  services  that  have  to  make  trade-offs.  It  is  not  an  AT&L  role. 

Better  Buying  Power  guidance  from  2012  also  called  for  portfolio-based 
analyses  to  identify  potential  redundancy  and  eliminate  unnecessary 
duplication;  however  these  analyses  have  not  been  conducted  (see  table 
10).  AT&L  was  to  conduct  these  analyses  for  ACAT  I  programs  in  support 
of  all  acquisition  program  decisions,  including  at  requirements  decision 
points,  and  during  annual  program  and  budget  review  processes.^  The 
military  services  were  to  conduct  these  analyses  for  lower  dollar 
acquisition  programs.  The  most  recent  version  of  the  Better  Buying  Power 


^Office  of  the  Under  Secretary  of  Defense,  Acquisition,  Technology,  and  Logistics 
Memorandum:  “Better  Buying  Power  2.0:  Continuing  the  Pursuit  for  Greater  Efficiency  and 
Productivity  in  Defense  Spending”  (Nov.  13,  2012)  and  Office  of  the  Under  Secretary  of 
Defense,  Acquisition,  Technology,  and  Logistics  Memorandum:  “Implementation  Directive 
for  Better  Buying  Power  2.0  -  Achieving  Greater  Efficiency  and  Productivity  in  Defense 
Spending”  (Apr.  24,  2013). 
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guidance  still  contains  the  goal  of  eliminating  redundancy  within 
warfighter  portfolios,  but  it  is  no  longer  an  area  of  emphasis. 

Table  10:  Better  Buying  Power  Activities  Related  to  Identifying  Redundancies  and  Their  Implementation  Status 

Better  Buying  Power  activity 

Description 

Implementation  status 

Enterprise-level  assessment  of 
redundancy  within  portfolios 
(2.0) 

The  Assistant  Secretary  of  Defense  for  Acquisition 
(ASD(A))  will  assess  redundancy  within  portfolios  in 
support  of  all  acquisition  and  program  decisions  during 
annual  program  and  budget  review  processes. 

Not  implemented.  ASD(A)  stated  that 
resource  limitations  and  lack  of  data 
prevent  implementation. 

Component-level  assessment 
of  redundancy  across 
portfolios  (2.0) 

Component  Acquisition  Executives  will  work  together 
as  a  group  to  consider  redundancy  across  Service  and 
Component  portfolios  as  they  review  Acquisition 
Category  II  and  III  programs  for  acquisition  decisions. 

Not  implemented.  Military  services 
determined  that  existing  ad  hoc  working 
groups  are  sufficient. 

Source:  GAO  assessment  of  Better  Buying  Power  guidance  and  implementation  status.  |  GAO-15-466 
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THE  JOINT  STAFF 

WASHINGTON,  DC 

30  July  2015 


Reply  ZIP  Code: 
20318-0300 


Mr.  Michael  J.  Sullivan 

Director.  Acquisition  and  Sourcing  Management 
U.S.  Government  Accountability  Office 
441  G  Street.  NW 
Washington  DC  20548 

Dear  Mr.  Sullivan: 

This  is  the  Department  of  Defense  response  to  the  L'.S.  Government  Accountability 
Office  (GAO)  Draft  Report.  GAO- 15-466,  “Weapon  System  Acquisitions:  Opportunities  Exist  to 
Improve  the  Department  of  Defense's  Portfolio  Management,”  dated  19  June  2015  (GAO  Code 
121179). 

The  Department  is  providing  official  written  comments  for  inclusion  in  the  report. 

Sincerely, 


vjvl. 

JACQURLINE  D.  VAN  OVOST.  Maj  Gen,  USAF 
Vice  Dirktor,  Joint  Staff 

Enclosure: 

As  stated 
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GAO-15-466  (GAO  CODE  121179) 

“WEAPON  SYSTEM  ACQUISITIONS:  OPPORTUNITIES  EXIST  TO  IMPROVE  THE 
DEPARTMENT  OF  DEFENSE’S  PORTFOLIO  MANAGEMENT” 

DEPARTMENT  OF  DEFENSE  COMMENTS 
TO  THE  GAO  RECOMMENDATIONS 


GAO  DRAFT  REPORT  RECOMMENDATION  1:  To  provide  a  solid  foundation  for  future 
portfolio  management  efforts  at  the  enterprise  level,  the  GAO  recommends  that  the  Secretary  of 
Defense  revise  Department  of  Defense  Directive  (DoDD)  7045.20,  “Capability  Portfolio 
Management,”  in  accordance  with  best  practices  and  promote  the  development  of  better  tools  to 
enable  more  integrated  portfolio  reviews  and  analyses  of  weapon  system  investments.  Key 
elements  of  this  recommendation  would  include  the  following; 

•  Designating  the  Deputy  Secretary  of  Defense  (DepSecDef)  or  some  appropriate  delegate 
responsibility  for  implementing  the  policy  and  overseeing  portfolio  management  in  the 
Department  of  Defense; 

•  Requiring  annual  enterprise-level  portfolio  reviews  that  incorporate  key  portfolio  review 
elements,  including  information  from  the  requirements,  acquisition,  and  budget 
processes; 

•  Directing  the  Joint  Staff,  the  Office  of  the  Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics  (OUSD(AT&L)),  and  the  Office  of  Cost  Assessment  and 
Program  Evaluation  (CAPE)  to  collaborate  on  their  data  needs  and  develop  a  formal 
implementation  plan  for  meeting  those  needs  either  by  building  on  the  database  the  Joint 
Staff  is  developing  for  its  analysis  or  investing  in  new  analytical  tools;  and 

•  Incorporating  lessons  learned  from  military  service  portfolio  reviews  and  portfolio 
management  activities,  such  as  using  multiple  risk  and  funding  scenarios  to  assess  needs 
and  reevaluate  priorities. 

DoD  DRAFT  RESPONSE  (JOINT  STAFF/OUSD(AT&L)):  Partially  concur.  While  the 
Department  of  Defense  agrees  with  the  intent  of  further  improving  portfolio  management  across 
the  DoD  enterprise  and  concurs  with  the  recommendation  to  further  develop  portfolio 
management  tools,  ensure  access  to  authoritative  data,  and  incorporate  lessons  learned  by  others 
performing  portfolio  management,  the  other  aspects  of  the  recommendation  are  redundant  to,  and 
will  serve  to  conflict  with,  other  processes  and  activities  in  place  to  perform  robust  portfolio 
management  both  from  a  day-to-day  management  and  annual  review  perspective. 

•  DoDD  7045.2,  “Capability  Portfolio  Management,”  should  be  rescinded  rather  than 
revised,  with  stakeholders  directed  to  participate  in  the  robust  portfolio  management 
outlined  in  associated  Joint  Capabilities  Integration  and  Development  System  (JCIDS), 
Defense  Acquisition  System  (DAS),  and  Planning,  Programming,  Budgeting,  and 
Execution  (PPBE)  processes.  In  light  of  revisions  to  the  JCIDS  and  DAS  in  early  2015, 
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the  means  by  which  these  processes  integrate  has  been  vastly  improved  over  earlier 
guidance. 

•  The  Deputy  Secretary  of  Defense  already  makes  the  ultimate  decisions  on  portfolio 
management  as  the  signatory  to  the  Resource  Decision  Memorandums  (RMDs)  that  are 
promulgated  at  the  end  of  the  Program  Budget/Review  (PBR),  and  a  new  designation  of 
this  oversight  role  is  not  necessary.  As  iong  as  the  PBR  efforts  leading  to  the  RMD  are 
fully  informed  by  capability  requirement  portfolios  (overseen  by  the  Vice  Chairman  of 
the  Joint  Chiefs  of  Staff  (VCJCS),  as  the  chair  of  the  Joint  Requirements  Oversight 
Council  (JROC))  and  by  materiel  acquisition  and  science  and  technology  development 
efforts  (overseen  by  the  USD(AT&L)  as  the  Defense  Acquisition  Executive),  then  the 
RMD  captures  an  annual  set  of  decisions  providing  direction  to  adjust  the  portfolios  to 
best  serve  the  Joint  Force. 

•  Requiring  additional  enterprise-level  portfolio  reviews  would  be  redundant  to  annual 
reviews  already  eonducted  through  existing  Departmental  processes  that  adjust  to  meet 
changing  threats,  technologies,  fiscal  realities,  and  legislative  direction. 

o  As  part  of  JCIDS  (requirements),  an  annual  Capability  Gap  Assessment  (CGA) 
reviewing  Combatant  Command  Integrated  Priority  List  submissions  together 
with  other  issues  impacting  the  capability  portfolios  serves  as  an  annual  portfolio 
review.  The  VCJCS  has  recently  released  updated  requirements  guidance  in 
CJCS  Instruction  (CJCSI)  5123.01,  CJCSI  3170.01,  and  the  JCIDS  Manual,  and 
has  approved  updates  to  the  Joint  Capability  Areas  (JCAs)  that  serve  as  the 
framework  for  portfolio  management.  These  updates  also  included 
implementation  of  the  Capability-Mission  Lattice,  whieh  depicts 
interrelationships  of  the  various  Department  management  constructs. 

o  By  its  nature,  the  CGA  considers  the  entirety  of  existing  and  new  capability 
solutions  under  development  through  DAS  (acquisition).  Additionally, 
OUSD(AT&L)  leads  comprehensive  reviews  of  long-term  science  and  technology 
investment  plans  to  help  frame  future  capability  portfolios.  The  USD(AT&L)  has 
recently  released  updated  acquisition  guidance  in  DoD  Instruction  5000.02  and 
continues  to  align  and  streamline  acquisition  activities  through  the  Better  Buying 
Power  3.0  initiatives.  The  Assistant  Secretary  of  Defense  for  Acquisition  has  also 
realigned  offices  and  responsibilities  to  better  align  with  the  JCAs  used  in  JCIDS 
for  portfolio  management. 

o  The  output  of  the  CGA  is  intended  to  inform  the  Service  Program  Objective 
Memorandum  (POM)  builds,  which  are  then  reviewed  for  consistency  with  the 
CGA  during  the  CAPE-led  PBR.  The  DepSecDef-signed  RMD  produced  at  the 
conclusion  of  the  PBR  is  the  means  to  direct  necessary  changes  to  manage  the 
portfolios.  In  the  last  3  years,  the  Strategic  Choices  and  Management  and 
Strategic  Choices  Reviews,  with  participation  from  across  the  Department,  have 
also  augmented  and  informed  the  PBR  process  through  focused  “deep  dive” 
assessments  of  the  Department’s  highest  priority  issues.  Finally,  the  Deputy 
Secretary  of  Defense  has  directed  a  shift  to  the  left  of  PPBE  activities  to  allow 
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more  time  for  robust  assessment  and  shaping  of  DoD  inputs  into  the  President’s 
budget. 

•  Continuing  to  advance  the  collaboration  efforts  between  the  Joint  Staff,  OUSD(AT&L), 
and  CAPE  in  terms  of  sharing  authoritative  data  and  developing  consistent  means  of 
analyzing  data  is  a  key  aspect  of  improving  portfolio  management.  While  the  recent 
JCIDS  changes  have  mandated  specific  structured  data  for  new  capability  requirement 
documents  as  a  means  to  better  enable  portfolio  management,  the  population  of  legacy 
capability  requirement  and  capability  solution  data  for  legacy  systems  is  necessary.  This 
“catch-up”  of  legacy  data  is  perhaps  the  lowest  hanging  fruit  to  improve  these  efforts, 
with  additional  analytical  tools  to  be  considered  as  a  follow-on  or  parallel  effort.  Another 
key  effort  underway  is  the  Acquisition,  Intelligence,  and  Requirements  integration,  which 
increases  robust  collaboration  between  these  three  communities  to  develop  a  common 
understanding  of  adversary  threat  (for  the  setting  of  capability  requirements  in  JCIDS  and 
the  development  of  capability  solutions  in  DAS),  and  to  ensure  the  appropriate 
Intelligence  Mission  Data  is  available  to  support  our  most  advanced  weapon  systems. 

•  Reviewing  lessons  learned  from  Services,  other  government  agencies,  and  industry 
portfolio  management  efforts  will  continue  to  shape  efforts  to  refine  DoD  portfolio 
management  efforts.  As  each  part  of  the  Department’s  overall  portfolio  management  is 
optimized  for  the  authorities  of  the  responsible  organization,  better  integration  of  the 
Services’  distinct  portfolio  management  approaches  and  lessons  learned  into  current 
DoD-wide  activities,  particularly  in  areas  addressing  budget  vagaries  and  risk,  would 
enhance  the  Department’s  ability  to  present  a  comprehensive  and  coherent  investment 
plan  to  execute  the  National  Defense  Strategy. 

GAO  DRAFT  REPORT  RECOMMENDATION  2;  To  help  ensure  the  Military  Services’ 
portfolio  reviews  are  conducted  regularly  and  effectively  integrate  information  from  the 
requirements,  acquisition,  and  budget  communities,  the  GAO  recommends  that  the  Secretary  of 
Defense  direct  the  Secretaries  of  the  Army,  Navy,  and  Air  Force  to  update  or  develop  policies 
that  require  them  to  conduct  annual  portfolio  reviews  that  incorporate  key  portfolio  review 
elements,  including  information  from  the  requirements,  acquisition,  and  budget  processes. 

DoD  DRAFT  RESPONSE  (JOINT  STAFF/OUSD(AT&L)):  Partially  concur.  The 
Department  of  Defense  agrees  that  Services  should  perform  both  day-to-day  management  and 
annual  reviews  of  their  respective  parts  of  the  DoD  portfolios,  but  believe  this  is  already 
accomplished  through  the  Service  POM  builds  and  submission  to  the  Office  of  the  Secretary  of 
Defense  for  review  in  PBR.  The  existing  process  also  provides  the  Department  of  Defense  the 
opportunity  to  review,  and  perhaps  enhance,  guidance  provided  to  the  Services  to  ensure 
maximum  alignment  between  the  Service  internal  portfolio  management  and  the  contribution  of 
the  service  capabilities  to  the  DoD  portfolios  and  the  needs  of  the  Joint  Force.  In  fact, 
USD(AT&L)  routinely  conducts  preliminary  reviews  of  Services’  POMs  prior  to  CAPE  analysis. 
Greater  alignment  in  this  area  has  potential  to  reduce  the  number  of  Combatant  Command  issues 
to  be  discussed  in  the  annual  CGA,  and  potential  to  reduce  the  number  of  issue  papers 
recommending  changes  to  the  Service  POMs  to  be  discussed  during  PBR. 
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